State of science: mental workload in ergonomics

An Investigation of Speech Features, Plant System Alarms, and Operator-System Interaction for the Classification of Operator Cognitive Workload During Dynamic Work

OBJECTIVE

To investigate speech features, human-machine alarms, and operator-system interaction for the estimation of cognitive workload in full-scale realistic simulated scenarios.

BACKGROUND

Theories and models of cognitive workload are critical for the design and evaluation of human-machine systems. Unfortunately, there are very few nonintrusive cognitive workload measures available for realistic dynamic human-machine interaction.

METHOD

The study was conducted in a full-scope control room research simulator of an advanced nuclear reactor. Six crews, each consisting of three operators, participated in 12 scenarios. The operators rated their workload every second minute. Machine learning algorithms were trained to estimate operators' workload based on crew communication, operator-system interaction, and system alarms.

RESULTS

Random Forest (RF) utilizing speech and system features achieved an accuracy of 67% on test data. Utilizing speech features only, the accuracy achieved was 63%. The most important speech features were pitch, amplitude, and articulation rate. A 61% accuracy was achieved when alarms and operator-system interaction features were used. The most important features were the number of alarms and amount of operator-system interaction. Accuracy for algorithms trained for each operator ranged from 39% to 98%, with an average of 72%. For a majority of analyses performed, RF and extreme gradient boosting (XGB) outperformed other algorithms.

CONCLUSION

The results demonstrate that the features investigated and machine learning models developed provide a potential for the dynamic nonintrusive measurement of cognitive workload.

APPLICATION

The approach presented can be developed for nonintrusive workload measurement in real-world human-machine applications, simulator-based training, and research.

The mediating effect of workers' situation awareness on the relationship between work-related factors and human error: a path analysis approach

Situation awareness is the main reason behind different patterns of unsafe behavior among workers and can play a mediating role in the relationship between predictive situational variables and human error. This questionnaire-based study carried out with Iranian workers investigated the direct and indirect effects of work pressure, mental workload, human-systems interaction and environmental distractions on three types of human error (i.e., slips, lapses and mistakes). The results of path analysis showed that, based on model fit indexes, the model is appropriately fit (χ² / df = 3, comparative fit index = 099, Tucker-Lewis index = 0.99, root mean square error of approximation = 026). It was also observed that the effects of four independent variables on all three types of human error are indirect and mediated by situation awareness. Consequently, it is confirmed that situation awareness plays a key role in the occurrence of human error and is a sharp-end causal factor for human error in industrial workplaces.

Research Outputs in Ergonomics and Human Factors Engineering: A bibliometric and co-word analysis of content and contributions

This paper analyzes research outputs in Ergonomics and Human Factors Engineering (EHFE), revealing its intellectual structure via bibliometric techniques, co-word, network analysis, and science visualization tools. Population comprises of 23,472 records published during 2000-2018, in 19 core journals of Human Factors and Ergonomics subject category in Scopus. The findings showed that in EHFE fields "the United States", "University of Central Florida" and "Stanton NA" were the most productive country, university and author, respectively. It was also found that "Ergonomics" was the most frequent keyword and "Ergonomics and Human Factor" were the most frequent co-occurred keywords in EHFE documents. Hierarchical cluster analysis led to the creation of eight topical clusters, including among others "Biomechanics ergonomics", "Work-related MSDs & work system design", and "Performance". The results indicated that "Biomechanics ergonomics" was a well-matured cluster while "Human machine interaction", "Ergonomics design", "Macro-ergonomics", and "Cognitive ergonomics" were found to be emerging or declining clusters.

Reference values and subscale patterns for the task load index (TLX): a meta-analytic review

The Task Load Index (TLX) is the predominant instrument for self-reporting workload. On the basis of a meta-analytic review of 556 studies, this paper supplies reference values for TLX and its six subscales across domains, technologies, regions, and real-life/lab settings. Across domains, TLX spans mean values from 35 for leisure to 56 for manual labour. TLX tends to be driven upward by the subscales of mental demand and effort and downward by the subscales of physical demand and frustration. For technologies, handheld devices are associated with lower TLX, possibly because they are simpler and more task-specific. TLX also varies across regions in that it is higher for studies in Asia than in Europe and North America. This variation is only partly explained by co-variation in domains. Furthermore, TLX is higher and its subscales more inter-correlated when it is studied in real-life rather than lab settings. Practitioner summary: Practitioners can use the reference values supplied in this paper to benchmark their TLX measurements against those from the corpus of TLX research. Furthermore, the reported subscale patterns add to the diagnostic power of the TLX instrument. Abbreviations: TLX: task load index; MD: mental demand; PD: physical demand; TD: temporal demand; EF: effort; PE: performance; FR: frustration.

Using Past and Present Indicators of Human Workload to Explain Variance in Human Performance

Cognitive workload is assumed to influence performance due to resource competition. However, there is a lack of evidence for a direct relationship between changes in workload within an individual over time and changes in that individual's performance. We collected performance data using a multiple object-tracking task in which we measured workload objectively in real-time using a modified detection response task. Using a multi-level Bayesian model controlling for task difficulty and past performance, we found strong evidence that workload both during and preceding a tracking trial was predictive of performance, such that higher workload led to poorer performance. These negative workload-performance relationships were remarkably consistent across individuals. Importantly, we demonstrate that fluctuations in workload independent from the task demands accounted for significant performance variation. The outcomes have implications for designing real-time adaptive systems to proactively mitigate human performance decrements, but also highlight the pervasive influence of cognitive workload more generally.

Machine learning-based analysis of operator pupillary response to assess cognitive workload in clinical ultrasound imaging

Introduction

Pupillometry, the measurement of eye pupil diameter, is a well-established and objective modality correlated with cognitive workload. In this paper, we analyse the pupillary response of ultrasound imaging operators to assess their cognitive workload, captured while they undertake routine fetal ultrasound examinations. Our experiments and analysis are performed on real-world datasets obtained using remote eye-tracking under natural clinical environmental conditions.

Methods

Our analysis pipeline involves careful temporal sequence (time-series) extraction by retrospectively matching the pupil diameter data with tasks captured in the corresponding ultrasound scan video in a multi-modal data acquisition setup. This is followed by the pupil diameter pre-processing and the calculation of pupillary response sequences. Exploratory statistical analysis of the operator pupillary responses and comparisons of the distributions between ultrasonographic tasks (fetal heart versus fetal brain) and operator expertise (newly-qualified versus experienced operators) are performed. Machine learning is explored to automatically classify the temporal sequences into the corresponding ultrasonographic tasks and operator experience using temporal, spectral, and time-frequency features with classical (shallow) models, and convolutional neural networks as deep learning models.

Results

Preliminary statistical analysis of the extracted pupillary response shows a significant variation for different ultrasonographic tasks and operator expertise, suggesting different extents of cognitive workload in each case, as measured by pupillometry. The best-performing machine learning models achieve receiver operating characteristic (ROC) area under curve (AUC) values of 0.98 and 0.80, for ultrasonographic task classification and operator experience classification, respectively.

Conclusion

We conclude that we can successfully assess cognitive workload from pupil diameter changes measured while ultrasound operators perform routine scans. The machine learning allows the discrimination of the undertaken ultrasonographic tasks and scanning expertise using the pupillary response sequences as an index of the operators’ cognitive workload. A high cognitive workload can reduce operator efficiency and constrain their decision-making, hence, the ability to objectively assess cognitive workload is a first step towards understanding these effects on operator performance in biomedical applications such as medical imaging.

•

Machine learning-based pupillary response analysis is performed to assess operator cognitive workload in clinical ultrasound.

•

A systematic multi-modal data analysis pipeline is proposed using eye-tracking, pupillometry, and sonography data science.

•

Pertinent challenges of natural or real-world clinical datasets are addressed.

•

Pupillary responses around event triggers, different ultrasonographic tasks, and different operator experiences are studied.

•

Machine learning models are learnt to classify undertaken tasks or operator expertise from pupillometric time-series data.

Online Multimodal Inference of Mental Workload for Cognitive Human Machine Systems

With increasingly higher levels of automation in aerospace decision support systems, it is imperative that the human operator maintains a high level of situational awareness in different operational conditions and a central role in the decision-making process. While current aerospace systems and interfaces are limited in their adaptability, a Cognitive Human Machine System (CHMS) aims to perform dynamic, real-time system adaptation by estimating the cognitive states of the human operator. Nevertheless, to reliably drive system adaptation of current and emerging aerospace systems, there is a need to accurately and repeatably estimate cognitive states, particularly for Mental Workload (MWL), in real-time. As part of this study, two sessions were performed during a Multi-Attribute Task Battery (MATB) scenario, including a session for offline calibration and validation and a session for online validation of eleven multimodal inference models of MWL. The multimodal inference model implemented included an Adaptive Neuro Fuzzy Inference System (ANFIS), which was used in different configurations to fuse data from an Electroencephalogram (EEG) model’s output, four eye activity features and a control input feature. The results from the online validation of the ANFIS models demonstrated that five of the ANFIS models (containing different feature combinations of eye activity and control input features) all demonstrated good results, while the best performing model (containing all four eye activity features and the control input feature) showed an average Mean Absolute Error (MAE) = 0.67 ± 0.18 and Correlation Coefficient (CC) = 0.71 ± 0.15. The remaining six ANFIS models included data from the EEG model’s output, which had an offset discrepancy. This resulted in an equivalent offset for the online multimodal fusion. Nonetheless, the efficacy of these ANFIS models could be seen with the pairwise correlation with the task level, where one model demonstrated a CC = 0.77 ± 0.06, which was the highest among all the ANFIS models tested. Hence, this study demonstrates the ability for online multimodal fusion from features extracted from EEG signals, eye activity and control inputs to produce an accurate and repeatable inference of MWL.

Using Eye Tracking for Measuring Cognitive Workload During Clinical Simulations: Literature Review and Synthesis

High-fidelity clinical simulations can be used by clinicians to acquire technical (physical ability and knowledge) and non-technical (cognitive and social processes) skills. Excessive cognitive workload contributes to medical errors because of the impact on both technical and non-technical skills. Many studies measure cognitive workload with psychometric instruments that limit the assessment of cognitive workload to a single time period and may involve response bias. Using eye tracking to measure task-evoked pupillary responses allows the measurement of changes in pupil diameter related to the cognitive workload associated with a specific activity. Incorporating eye tracking with high-fidelity clinical simulations provides a reliable and continuous assessment of cognitive workload. The purpose of this literature review is to summarize the use of eye-tracking technology to measure cognitive workload of healthcare providers to generate evidence-based guidelines for measuring cognitive workload during high-fidelity clinical simulations. What this manuscript adds to the body of literature is a summary of best practices related to the different methods of measuring cognitive workload, benefits and limitations of using eye tracking, and high-fidelity clinical simulation design considerations for successful integration of eye tracking.

FINE-GRAINED AND MULTI-SCALE MOTIF FEATURES FOR CROSS-SUBJECT MENTAL WORKLOAD ASSESSMENT USING BI-LSTM

Mental workload (MW) assessment is crucial for understanding human mental state. Cross-subject MW analysis based on electroencephalogram (EEG) signals is an important way. In this paper, a fine-grained and multi-scale motif (FGMSM) features extraction method is proposed, and the proposed features together with original EEG data are used as the input of bidirectional long short-term memory (Bi-LSTM) to evaluate the cross-subject mental workload. First, the EEG signal of each channel is decomposed based on improved complete ensemble empirical mode decomposition with adaptive noise (ICEEMDAN) algorithm. Second, for the motif structure consisting of three nodes, multi-scale detection is carried out in each intrinsic mode function, and the proportion of each motif structure is extracted as the newly extracted features. Then, the statistical differences of the extracted features between different MW levels are analyzed by using the [Formula: see text]-test, and the features with statistical differences are selected for the cross-subject MW assessment. Finally, based on the public dataset with 26 subjects, Bi-LSTM and a variety of machine learning algorithms are used to classify the levels of cross-subject MW. The results show that the Bi-LSTM classification method with the original EEG data and the proposed features show the most positive results. Therefore, the FGMSM features proposed in this paper with Bi-LSTM provide a new technique for the assessment of cross-subject MW based on EEG signals.

The Index of Cognitive Activity - Eligibility for task-evoked informational strain and robustness towards visual influences

Various researchers have proposed pupillometric indicators to assess a person's cognitive strain. However, to distinguish the variation of pupil light response from psychosensory pupil response in experimental field conditions is a challenge. The Index of Cognitive Activity (ICA) addresses this problem by wavelet separation. This research investigates the ICA's sensitivity for multiple level task-evoked cognitive activity and visual influences concerning informational work tasks. Objective and subjective measures assessed cognitive strain of participants (N = 22) during various tasks. In a first experiment, mental arithmetic tasks were used to induce different levels of cognitive activity. In a second experiment, influences of screen polarity and presentation of information were investigated (N = 18). The results indicate that eye metrics are rarely sensitive to slight variations in task difficulty. Moreover, the ICA is likely to be sensitive towards constant screen illumination and shows tendencies regarding changes in displayed information. Possible ramifications for the objective assessment of cognitive strain are discussed.

Wearable Technologies for Mental Workload, Stress, and Emotional State Assessment during Working-Like Tasks: A Comparison with Laboratory Technologies

The capability of monitoring user’s performance represents a crucial aspect to improve safety and efficiency of several human-related activities. Human errors are indeed among the major causes of work-related accidents. Assessing human factors (HFs) could prevent these accidents through specific neurophysiological signals’ evaluation but laboratory sensors require highly-specialized operators and imply a certain grade of invasiveness which could negatively interfere with the worker’s activity. On the contrary, consumer wearables are characterized by their ease of use and their comfortability, other than being cheaper compared to laboratory technologies. Therefore, wearable sensors could represent an ideal substitute for laboratory technologies for a real-time assessment of human performances in ecological settings. The present study aimed at assessing the reliability and capability of consumer wearable devices (i.e., Empatica E4 and Muse 2) in discriminating specific mental states compared to laboratory equipment. The electrooculographic (EOG), electrodermal activity (EDA) and photoplethysmographic (PPG) signals were acquired from a group of 17 volunteers who took part to the experimental protocol in which different working scenarios were simulated to induce different levels of mental workload, stress, and emotional state. The results demonstrated that the parameters computed by the consumer wearable and laboratory sensors were positively and significantly correlated and exhibited the same evidences in terms of mental states discrimination.

Effects of whole body vibration and backrest angle on perceived mental workload and performance

Mental Workload (MWL) and human performance are widely contributing concepts in human factors. The objective of the current study is to investigate the perceived MWL and human performance during whole-body vibration (WBV) exposure while seated at different backrest angles. Nineteen healthy male participants completed both the NASA-TLX and rating scale mental effort (RSME) after performing two difficulty levels of computerized dual tasks. The participants' performance was measured in these conditions while seated with a backrest angle of 100 and 120 degrees and exposed to WBV (intensity: 0.5 m/s²; frequency 3-20 Hz) for 5 minutes. No significant effect on performance or perceived MWL (p<0.05) was found when changes were made to the backrest angles. Exposure to WBV under two backrest angles increased mental demand (p=0.04), effort (p=0.03) and frustration (p=0.03) and negatively affected human performance (p<0.05). The present study showed that exposure to WBV could be an important variable for designing work environments that require a high level of performance and mental demand while seated. However, the findings exhibited no association between inclining backrest angle and human performance or perceived MWL.

Ultra-Short Window Length and Feature Importance Analysis for Cognitive Load Detection from Wearable Sensors

Human cognitive capabilities are under constant pressure in the modern information society. Cognitive load detection would be beneficial in several applications of human–computer interaction, including attention management and user interface adaptation. However, current research into accurate and real-time biosignal-based cognitive load detection lacks understanding of the optimal and minimal window length in data segmentation which would allow for more timely, continuous state detection. This study presents a comparative analysis of ultra-short (30 s or less) window lengths in cognitive load detection with a wearable device. Heart rate, heart rate variability, galvanic skin response, and skin temperature features are extracted at six different window lengths and used to train an Extreme Gradient Boosting classifier to detect between cognitive load and rest. A 25 s window showed the highest accury (67.6%), which is similar to earlier studies using the same dataset. Overall, model accuracy tended to decrease as the window length decreased, and lowest performance (60.0%) was observed with a 5 s window. The contribution of different physiological features to the classification performance and the most useful features that react in short windows are also discussed. The analysis provides a promising basis for future real-time applications with wearable sensors.

Multimodal Fusion for Objective Assessment of Cognitive Workload: A Review

Considerable progress has been made in improving the estimation accuracy of cognitive workload using various sensor technologies. However, the overall performance of different algorithms and methods remain suboptimal in real-world applications. Some studies in the literature demonstrate that a single modality is sufficient to estimate cognitive workload. These studies are limited to controlled settings, a scenario that is significantly different from the real world where data gets corrupted, interrupted, and delayed. In such situations, the use of multiple modalities is needed. Multimodal fusion approaches have been successful in other domains, such as wireless-sensor networks, in addressing single-sensor weaknesses and improving information quality/accuracy. These approaches are inherently more reliable when a data source is lost. In the cognitive workload literature, sensors, such as electroencephalography (EEG), electrocardiography (ECG), and eye tracking, have shown success in estimating the aspects of cognitive workload. Multimodal approaches that combine data from several sensors together can be more robust for real-time measurement of cognitive workload. In this article, we review the published studies related to multimodal data fusion to estimate the cognitive workload and synthesize their main findings. We identify the opportunities for designing better multimodal fusion systems for cognitive workload modeling.

Classification of Drivers' Workload Using Physiological Signals in Conditional Automation

The use of automation in cars is increasing. In future vehicles, drivers will no longer be in charge of the main driving task and may be allowed to perform a secondary task. However, they might be requested to regain control of the car if a hazardous situation occurs (i.e., conditionally automated driving). Performing a secondary task might increase drivers' mental workload and consequently decrease the takeover performance if the workload level exceeds a certain threshold. Knowledge about the driver's mental state might hence be useful for increasing safety in conditionally automated vehicles. Measuring drivers' workload continuously is essential to support the driver and hence limit the number of accidents in takeover situations. This goal can be achieved using machine learning techniques to evaluate and classify the drivers' workload in real-time. To evaluate the usefulness of physiological data as an indicator for workload in conditionally automated driving, three physiological signals from 90 subjects were collected during 25 min of automated driving in a fixed-base simulator. Half of the participants performed a verbal cognitive task to induce mental workload while the other half only had to monitor the environment of the car. Three classifiers, sensor fusion and levels of data segmentation were compared. Results show that the best model was able to successfully classify the condition of the driver with an accuracy of 95%. In some cases, the model benefited from sensors' fusion. Increasing the segmentation level (e.g., size of the time window to compute physiological indicators) increased the performance of the model for windows smaller than 4 min, but decreased for windows larger than 4 min. In conclusion, the study showed that a high level of drivers' mental workload can be accurately detected while driving in conditional automation based on 4-min recordings of respiration and skin conductance.

Mental workload of frontline nurses aiding in the COVID-19 pandemic: A latent profile analysis

Aims

To investigate the mental workload level of nurses aiding the most affected area during the Coronavirus disease 2019 (COVID‐19) pandemic and explore the subtypes of nurses regarding their mental workload.

Design

Cross‐sectional study.

Methods

A sample of 446 frontline nurses participated from March 8 to 19, 2020. A latent profile analysis was performed to identify clusters based on the six subscales of the Chinese version of the National Aeronautics and Space Administration Task Load Index. The differences among the classes and the variables including sociodemographic characteristics, psychological capital and coping style were explored.

Results

The level of mental workload indicates that the nurses had high self‐evaluations of their performance while under extremely intensive task loads. The following three latent subtypes were identified: ‘low workload & low self‐evaluation’ (8.6%); ‘medium workload & medium self‐evaluation’ (35.3%) and ‘high workload & high self‐evaluation’ (56.1%) (Classes 1, 2, and 3, respectively). Nurses with shared accommodations, fewer years of practice, junior professional titles, lower incomes, nonmanagement working positions, lower psychological capital levels and negative coping styles had a higher likelihood of belonging to Class 1. In contrast, senior nurses with higher psychological capital and positive coping styles were more likely to belong to Classes 2 and 3.

Conclusion

The characteristics of the ‘low workload & low self‐evaluation’ subtype suggest that attention should be paid to the work pressure and psychological well‐being of junior nurses. Further research on regular training program of public health emergency especially for novices is needed. Personnel management during public health events should be focused on the allocation between novice and senior frontline nurses.

Impact

This study addresses the level of mental workload of frontline nurses who aid in the most severe area of the COVID‐19 pandemic in China and delineates the characteristics of the subtypes of these nurses.

Attention-Dependent Physiological Correlates in Sleep-Deprived Young Healthy Humans

Cognitive functions could be specifically altered but masked from the unspecific effect of workload, a common factor affecting cognitive functions that modulate peripheral outputs. To identify workload-related and specific, task-dependent components, physiological correlates of cognitive functioning were derived by studying 15 healthy volunteers performing attentional tasks in baseline and post-sleep-deprivation conditions (one week interval). Sleep deprivation was introduced to increase workload. We performed recordings of heart pulse, facial temperature, and head movements during tasks assessing attentional network efficiency (ANT, Attentional Network Task; CCT, Continuous Compensatory Tracker) workload assessments after execution of tasks. Changes in cognitive and physiological indices were studied in both conditions; physiological correlates of cognitive performance were identified by correlating changes from baseline to post-sleep-deprivation condition of task indices with those of physiological measures after correction for between-conditions workload changes. We found that mental and physical demands of workload increased after sleep deprivation. We identified no changes in cognitive and physiological indices across conditions; specific physiological correlates of attentional systems, as indicated by the negative correlation between changes in ANT-alerting and changes in amplitude of head movements and the positive correlation between changes in CCT-speed indexing alertness and changes in facial temperature.

Influence of Increasing Task Complexity and Use of Informational Assistance Systems on Mental Workload

(1) Background: Cognitive aspects and complexity in modern manual mixed model assembly are increasing. To reduce mental workload (MWL), informational assistance systems are introduced. The influence of complexity and used assistance system on MWL should be investigated to further improve the implementation of such assistance systems. (2) Methods: Using a simulated close to real-life assembly task a 2 × 3 design was chosen, with two levels of assembly complexity (within subjects) and three different assistance systems (paper, Augmented Reality (AR)-glasses, tablet-between subjects). MWL was measured using either physiological response (electrocardiogram (ECG) and eye-tracking) or performance indicators. (3) Results: An influence of task complexity on MWL can be shown. Additionally, usability based differences between the used assistance systems become more evident with reference to the results of area of interest analysis. (4) Conclusions: Using a multi-modal measurement approach, it is possible to detect complexity-based differences in MWL. Additional research on validity and alignment is needed to further use these for (neuro-) ergonomic considerations and recommendations.

Evaluation of Fatigue and Workload among Workers Conducting Complex Manual Assembly in Manufacturing

OCCUPATIONAL APPLICATIONSWe conducted a study to evaluate fatigue and workload among workers performing complex assembly tasks. We investigate several predictors of fatigue, including subjective workload estimates, sleep duration, the shift being worked, and production levels. High levels of fatigue were reported in one-third of the shifts evaluated. The main predictors of high fatigue were workload estimates, working evening shifts, and baseline fatigue. Among the six dimensions of workload, only mental demand and frustration were predictors of high fatigue. Mental demand was also rated highest. Participants reported less than seven hours of sleep in 60% of the nights evaluated. These results suggest that managers and supervisors should consider cognitive workload as a key contributing factor to fatigue in complex manual assembly. Similarly, work schedule planning should consider shift duration, start times, and end times, because of the negative influence on fatigue and the potential disruptions on sleep among workers.

Development and validation of a behavioural video coding scheme for detecting mental workload in manual assembly

Manual assembly in the future Industry 4.0 workplace will put high demands on operators' cognitive processing. The development of mental workload (MWL) measures therefore looms large. Physiological gauges such as electroencephalography (EEG) show promising possibilities, but still lack sufficient reliability when applied in the field. This study presents an alternative measure with a substantial ecological validity. First, we developed a behavioural video coding scheme identifying 11 assembly behaviours potentially revealing MWL being too high. Subsequently, we explored its validity by analysing videos of 24 participants performing a high and a low complexity assembly. Results showed that five of the behaviours identified, such as freezing and the amount of part rotations, significantly differed in occurrence and/or duration between the two conditions. The study hereby proposes a novel and naturalistic method that could help practitioners to map and redesign critical assembly phases, and researchers to enrich validation of MWL-measures through measurement triangulation. Practitioner summary: Current physiological mental workload (MWL) measures still lack sufficient reliability when applied in the field. Therefore, we identified several observable assembly behaviours that could reveal MWL being too high. The results propose a method to map MWL by observing specific assembly behaviours such as freezing and rotating parts. Abbreviations: MWL: mental workload; EEG: electroencephalography; fNIRS: functional near infrared spectroscopy; AOI: area of interest; SMI: SensoMotoric Instruments, ETG: Eye-Tracking Glasses; FPS: frames per second; BORIS: Behavioral Observation Research Interactive Software; IRR: inter-rater reliability; SWAT: Subjective Workload Assessment Technique; NASA-TLX: National Aeronautics and Space Administration Task Load Index; EL: emotional load; DSSQ: Dundee Stress State Questionnaire; PHL: physical load; SBO: Strategisch Basis Onderzoek.

PASS: A Multimodal Database of Physical Activity and Stress for Mobile Passive Body/ Brain-Computer Interface Research

With the burgeoning of wearable devices and passive body/brain-computer interfaces (B/BCIs), automated stress monitoring in everyday settings has gained significant attention recently, with applications ranging from serious games to clinical monitoring. With mobile users, however, challenges arise due to other overlapping (and potentially confounding) physiological responses (e.g., due to physical activity) that may mask the effects of stress, as well as movement artifacts that can be introduced in the measured signals. For example, the classical increase in heart rate can no longer be attributed solely to stress and could be caused by the activity itself. This makes the development of mobile passive B/BCIs challenging. In this paper, we introduce PASS, a multimodal database of Physical Activity and StresS collected from 48 participants. Participants performed tasks of varying stress levels at three different activity levels and provided quantitative ratings of their perceived stress and fatigue levels. To manipulate stress, two video games (i.e., a calm exploration game and a survival game) were used. Peripheral physical activity (electrocardiography, electrodermal activity, breathing, skin temperature) as well as cerebral activity (electroencephalography) were measured throughout the experiment. A complete description of the experimental protocol is provided and preliminary analyses are performed to investigate the physiological reactions to stress in the presence of physical activity. The PASS database, including raw data and subjective ratings has been made available to the research community at http://musaelab.ca/pass-database/. It is hoped that this database will help advance mobile passive B/BCIs for use in everyday settings.

Eye-Tracking Metrics Predict Perceived Workload in Robotic Surgical Skills Training

OBJECTIVE

The aim of this study is to assess the relationship between eye-tracking measures and perceived workload in robotic surgical tasks.

BACKGROUND

Robotic techniques provide improved dexterity, stereoscopic vision, and ergonomic control system over laparoscopic surgery, but the complexity of the interfaces and operations may pose new challenges to surgeons and compromise patient safety. Limited studies have objectively quantified workload and its impact on performance in robotic surgery. Although not yet implemented in robotic surgery, minimally intrusive and continuous eye-tracking metrics have been shown to be sensitive to changes in workload in other domains.

METHODS

Eight surgical trainees participated in 15 robotic skills simulation sessions. In each session, participants performed up to 12 simulated exercises. Correlation and mixed-effects analyses were conducted to explore the relationships between eye-tracking metrics and perceived workload. Machine learning classifiers were used to determine the sensitivity of differentiating between low and high workload with eye-tracking features.

RESULTS

Gaze entropy increased as perceived workload increased, with a correlation of .51. Pupil diameter and gaze entropy distinguished differences in workload between task difficulty levels, and both metrics increased as task level difficulty increased. The classification model using eye-tracking features achieved an accuracy of 84.7% in predicting workload levels.

CONCLUSION

Eye-tracking measures can detect perceived workload during robotic tasks. They can potentially be used to identify task contributors to high workload and provide measures for robotic surgery training.

APPLICATION

Workload assessment can be used for real-time monitoring of workload in robotic surgical training and provide assessments for performance and learning.

Recognition of cognitive load with a stacking network ensemble of denoising autoencoders and abstracted neurophysiological features

The safety of human-machine systems can be indirectly evaluated based on operator's cognitive load levels at each temporal instant. However, relevant features of cognitive states are hidden behind in multiple sources of cortical neural responses. In this study, we developed a novel neural network ensemble, SE-SDAE, based on stacked denoising autoencoders (SDAEs) which identify different levels of cognitive load by electroencephalography (EEG) signals. To improve the generalization capability of the ensemble framework, a stacking-based approach is adopted to fuse the abstracted EEG features from activations of deep-structured hidden layers. In particular, we also combine multiple K-nearest neighbor and naive Bayesian classifiers with SDAEs to generate a heterogeneous classification committee to enhance ensemble's diversity. Finally, we validate the proposed SE-SDAE by comparing its performance with mainstream pattern classifiers for cognitive load evaluation to show its effectiveness.

Assessing cognitive load in adolescent and adult students using photoplethysmogram morphometrics

Compared to cardiac parameters and skin conductivities, the photoplethysmogram (PPG) recorded at fingertips and other parts near to peripheral nerve ends have been recently revealed to be yet another sensitive measure for cognitive load assessment. However, there is so far no research on measuring adolescents' cognitive load using physiological signals. A comprehensive study on the effects of PPG morphometrics over a cohort covering both adolescent and adult students is also absent. In this study, we analyze the morphological features of PPG on cognitive load assessment and compare them between adolescent and adult students. Experiments on two-level arithmetic tasks show that the PPG morphometrics reached the same level of significance on the effect of task difficulty/period as heart rate, and different morphological behaviors were also shown between adolescent and adult students during the cognitive task effects, which may imply their physiological differences across age. Physiological signals recorded by wearable devices are also found to be effective in measuring cognitive load.

Psychological Capital, Workload, and Burnout: What’s New? The Impact of Personal Accomplishment to Promote Sustainable Working Conditions

Research on burnout has traditionally focused on job demands, with less attention paid to protective factors. From the emerging and innovative area of psychology of sustainability and sustainable development, this study aimed to analyze the relationship that job demands (workload), and personal resources (psychological capital) have with burnout. The sample includes 517 workers from various professional sectors. Results of the structural equation analysis show that (1) psychological capital and workload are related to burnout, and (2) personal accomplishment is more a personal resource than burnout dimension. The results confirm the role of psychological capital as a protective factor for burnout. Finally, the importance of examining the relationship between the components of psychological capital, personal accomplishment, and positive emotions is pointed out due to the importance of organizations promoting the development of psychological strengths and resources to promote well-being and sustainable working conditions.

Specifying advantages of multi-modal cueing: Quantifying improvements with augmented tactile information

This work examines how tactile cues, encoded with azimuth and distance information, compare with visual and speech cues on performance and mental workload in a target detection task. Two experiments are reported using a simulated environment in which targets were presented at varying azimuth and distance locations. In the first experiment, participants engaged targets both while stationary and while in motion using tactile, visual, or speech cues. A no cueing control was included. In the second multi-modal experiment, participants completed the same task using cue pairings. Performance metrics consisted of hits, misses due to non-detection, misses due to inaccurate engagement, false alarms, response time, navigation errors as well as subjective ratings of mental workload scores were also collected. Results demonstrate the superiority of tactile cues as a means to communicate target location information either as a single modality or when paired with the two other cue types.

An Inland Shore Control Centre for Monitoring or Controlling Unmanned Inland Cargo Vessels

Augmenting the automation level of the inland waterway cargo transport sector, coupled with mechatronic innovation in this sector, could increase its competitiveness. This increase might potentially induce a sustainable paradigm shift in the road-dominated inland cargo transport sector. A key enabler of this envisaged shift may be an inland shore control centre (I-SCC) capable of remotely monitoring and controlling inland vessels. Accordingly, this study investigated the concept and design requirements to achieve an inland I-SCC that provides interaction services when supervising an unmanned surface vessel (USV). This I-SCC can help its operator to develop situational awareness and sensemaking. The conducted experiments offered insights into the performance of both the I-SCC system and its operator, and unlock research on the impact on ship sense and harmony when remotely controlling a USV. The Hull-To-Hull project extends the current I-SCC by providing enhanced motion control. This enhancement enables further performance insights and might improve the future monitoring of USVs. The successful I-SCC construction, the preliminary experiments, and the design-extension demonstrate that the I-SCC can serve as an experimental platform for both mechatronic innovation and human-automation integration research in the inland waterway sector, whilst additionally providing fruitful knowledge for adjacent research domains.

A Cyber-Physical-Human System for One-to-Many UAS Operations: Cognitive Load Analysis

The continuing development of avionics for Unmanned Aircraft Systems (UASs) is introducing higher levels of intelligence and autonomy both in the flight vehicle and in the ground mission control, allowing new promising operational concepts to emerge. One-to-Many (OTM) UAS operations is one such concept and its implementation will require significant advances in several areas, particularly in the field of Human-Machine Interfaces and Interactions (HMI2). Measuring cognitive load during OTM operations, in particular Mental Workload (MWL), is desirable as it can relieve some of the negative effects of increased automation by providing the ability to dynamically optimize avionics HMI2 to achieve an optimal sharing of tasks between the autonomous flight vehicles and the human operator. The novel Cognitive Human Machine System (CHMS) proposed in this paper is a Cyber-Physical Human (CPH) system that exploits the recent technological developments of affordable physiological sensors. This system focuses on physiological sensing and Artificial Intelligence (AI) techniques that can support a dynamic adaptation of the HMI2 in response to the operators' cognitive state (including MWL), external/environmental conditions and mission success criteria. However, significant research gaps still exist, one of which relates to a universally valid method for determining MWL that can be applied to UAS operational scenarios. As such, in this paper we present results from a study on measuring MWL on five participants in an OTM UAS wildfire detection scenario, using Electroencephalogram (EEG) and eye tracking measurements. These physiological data are compared with a subjective measure and a task index collected from mission-specific data, which serves as an objective task performance measure. The results show statistically significant differences for all measures including the subjective, performance and physiological measures performed on the various mission phases. Additionally, a good correlation is found between the two physiological measurements and the task index. Fusing the physiological data and correlating with the task index gave the highest correlation coefficient (CC = 0.726 ± 0.14) across all participants. This demonstrates how fusing different physiological measurements can provide a more accurate representation of the operators' MWL, whilst also allowing for increased integrity and reliability of the system.

Assessment of Implicit and Explicit Measures of Mental Workload in Working Situations: Implications for Industry 4.0

Nowadays, in the context of Industry 4.0, advanced working environments aim at achieving a high degree of human–machine collaboration. This phenomenon occurs, on the one hand, through the correct interpretation of operators’ data by machines that can adapt their functioning to support workers, and on the other hand, by ensuring the transparency of the actions of the system itself. This study used an ad hoc system that allowed the co-registration of a set of participants’ implicit and explicit (I/E) data in two experimental conditions that varied in the level of mental workload (MWL). Findings showed that the majority of the considered I/E measures were able to discriminate the different task-related mental demands and some implicit measures were capable of predicting task performance in both tasks. Moreover, self-reported measures showed that participants were aware of such differences in MWL. Finally, the paradigm’s ecology highlights that task and environmental features may affect the reliability of the various I/E measures. Thus, these factors have to be considered in the design and development of advanced adaptive systems within the industrial context.

Effects of Occupational Fatigue on Cognitive Performance of Staff From a Train Operating Company: A Field Study

Background

Occupational fatigue is a key issue in the rail industry that can endanger staff, passenger, and train safety. There is a need to demonstrate the relationship between workload, fatigue, and performance among rail staff.

Objective

The present study, conducted in the workplace in realistic situations, integrating both subjective and objective measurements, aimed at demonstrating the relationship between workload, fatigue, and cognitive performance with a rail staff sample.

Methods

The “After-Effect” technique was applied in the current study. Online diaries and cognitive performance tasks were used to assess the fatigue, work experiences, and performance of rail staff before and after work on the first and last days of one working week.

Results

Reported fatigue was greater after work on both the first and last day of the working week. There were large individual differences in the change in fatigue and workload ratings. Analysis of covariance with age and the pre-work performance score as covariates and the post-work performance score as the dependent variable showed that high levels of fatigue were associated with impaired performance on both the visual search and logical reasoning tasks. Workload had fewer effects on performance than fatigue.

Conclusion

This field study provided evidence for the relationship between work-related fatigue and performance impairment. The findings show the need for future work on predicting fatigue-related performance decrements, and the necessity of providing interventions and support so that the risk to safety can be reduced.

A Curvilinear Effect of Mental Workload on Mental Effort and Behavioral Adaptability: An Approach With the Pre-Ejection Period

OBJECTIVE

We tested Hancock and Szalma's mental workload model, which has never been experimentally validated at a global level with the measure of the pre-ejection period (PEP), an index of beta-adrenergic sympathetic impact.

BACKGROUND

Operators adapt to mental workload. When mental workload level increases, behavioral and physiological adaptability intensifies to reduce the decline in performance. However, if the mental workload exceeds an intermediate level, behavioral and physiological adaptability will decrease to protect individuals from excessive perturbations. This decrease is associated with a change in behavioral strategies and disengagement.

METHOD

The experimental task was a modified Fitts' task used in Hancock and Caird. Five levels of task difficulty were computed. Behavioral and physiological adaptability was indexed by the performance with speed-accuracy trade-off and PEP reactivity.

RESULTS

A curvilinear effect of task difficulty on PEP reactivity was significant, with high reactivity at the intermediate level but low reactivity at other levels. We observed a linear effect of task difficulty on error rate and a curvilinear effect on movement time. A decline in performance was noted up to the intermediate level, with a speed-accuracy trade-off above this level showing a faster movement time.

CONCLUSION

We observed for the first time behavioral and physiological adaptability as a function of mental workload.

APPLICATION

The results have important implications for the modeling of mental workload, particularly in the context of the performance-sensitive domain (car driving and air traffic control). They can help guide the design of human-computer interaction to maximize adaptive behavior, that is, the "comfort zone."

Using Machine Learning to Train a Wearable Device for Measuring Students' Cognitive Load during Problem-Solving Activities Based on Electrodermal Activity, Body Temperature, and Heart Rate: Development of a Cognitive Load Tracker for Both Personal and Classroom Use

Automated tracking of physical fitness has sparked a health revolution by allowing individuals to track their own physical activity and health in real time. This concept is beginning to be applied to tracking of cognitive load. It is well known that activity in the brain can be measured through changes in the body’s physiology, but current real-time measures tend to be unimodal and invasive. We therefore propose the concept of a wearable educational fitness (EduFit) tracker. We use machine learning with physiological data to understand how to develop a wearable device that tracks cognitive load accurately in real time. In an initial study, we found that body temperature, skin conductance, and heart rate were able to distinguish between (i) a problem solving activity (high cognitive load), (ii) a leisure activity (moderate cognitive load), and (iii) daydreaming (low cognitive load) with high accuracy in the test dataset. In a second study, we found that these physiological features can be used to predict accurately user-reported mental focus in the test dataset, even when relatively small numbers of training data were used. We explain how these findings inform the development and implementation of a wearable device for temporal tracking and logging a user’s learning activities and cognitive load.

The Overview of Mental Workload of Health Safety and Environment Workers in Oil and Gas Industry

 Introduction: PT. Pertamina RU IV Cilacap is a national oil and gas industry that has the largest refinery in Southeast Asia that manages raw materials for other refinery units in Indonesia to be reprocessed into fuel. The size of the refinery can illustrate the complexity of the system run by this unit. The HSE Unit in this company has a great responsibility to maintain the worker safety in the refinery. It is necessary to measure mental workload on HSE workers. This is because if the mental workload is not monitored, it can cause fatality, in this case, work accidents as a result of lack of supervision from HSE workers and mental PAK related to HSE workers resulting in excessive mental workload or property loss as an outcome of excessive mental workload. So, it is important to portray the HSE mental workload because of its responsibility to secure the health, safety and environmental issues. The purpose of this study was to describe the heavy mental workload on PT. Pertamina RU IV Cilacap. Method: This research was a quantitative descriptive designed study. Data were obtained by conducting interviews and using the NASA-tlx questionnaire on 71 respondents. Results: As many as 35 respondents had heavy mental workloads and 31 respondents had very heavy mental workloads. Conclusion: HSE workers at PT. Pertamina RU IV Cilacap has a heavy mental workload and even tends to be very heavy.Keywords: mental workload, NASA-tlx, oil and gas industry

Measurement and identification of mental workload during simulated computer tasks with multimodal methods and machine learning

This study attempted to multimodally measure mental workload and validate indicators for estimating mental workload. A simulated computer work composed of mental arithmetic tasks with different levels of difficulty was designed and used in the experiment to measure physiological signals (heart rate, heart rate variability, electromyography, electrodermal activity, and respiration), subjective ratings of mental workload (the NASA Task Load Index), and task performance. The indices from electrodermal activity and respiration had a significant increment as task difficulty increased. There were no significant differences between the average heart rate and the low-frequency/high-frequency ratio among tasks. The classification of mental workload using combined indices as inputs showed that classification models combining physiological signals and task performance can reach satisfying accuracy at 96.4% and an accuracy of 78.3% when only using physiological indices as inputs. The present study also showed that ECG and EDA signals have good discriminating power for mental workload detection. Practitioner summary: The methods used in this study could be applied to office workers, and the findings provide preliminary support and theoretical exploration for follow-up early mental workload detection systems, whose implementation in the real world could beneficially impact worker health and company efficiency. Abbreviations: NASA-TLX: the national aeronautics and space administration-task load index; ECG: electrocardiographic; EDA: electrodermal activity; EEG: electroencephalogram; LDA: linear discriminant analysis; SVM: support vector machine; KNN: k-nearest neighbor; ANNs: artificial neural networks; EMG: electromyography; PPG: photoplethysmography; SD: standard deviation; BMI: body mass index; DSSQ: dundee stress state questionnaire; ANOVA: analysis of variance; SC: skin conductance; RMS: root mean square; AVHR: the average heart rate; HR: heart rate; LF/HF: the ratio between the low frequencies band and the high frequency band; PSD: power spectral density; MF: median frequency; HRV: heart rate variability; BPNN: backpropagation neural network.

Early Alpha Reactivity is Associated with Long-Term Mental Fatigue Behavioral Impairments

The quantitative analysis of electroencephalogram (qEEG) is a suitable tool for mental fatigue (MF) assessment. Here, we evaluated the effects of MF on behavioral performance and alpha power spectral density (PSD) and the association between early alpha PSD reactivity and long-term behavioral MF impairments. Nineteen right-handed adults (21.21 ± 1.77 years old) had their EEG measured during five blocks of the visual oddball paradigm (~ 60 min). A paired t-test was used to compare first and last block values of cognitive performance and alpha PSD. The sample was divided into high (HAG) and low alpha group (LAG) by early alpha PSD median values. The behavioral performance of the HAG and LAG was compared across the blocks by a two-way ANOVA with repeated measures (groups and blocks). MF impairs general behavioral performance and increases alpha PSD. The HAG presents more behavioral impairment when compared to LAG across the task. Simple linear regression between early alpha PSD and behavioral performance across the task can predict 19 to 39% of variation in general behavior impairment by MF. In conclusion, MF induction impairs general behavioral and increases alpha PSD. The other finding was that higher alpha PSD reactivity is associated to higher long-term behavioral impairments of MF. This work contributes to existing knowledge of MF by providing evidence that the possibility of investigating early electrophysiological biomarkers to predict long-term MF impairments.

Pilots’ mental workload prediction based on timeline analysis

BACKGROUND:

The aircraft cockpit is a highly intensive human-computer interaction system, and its design directly affects flight safety.

OBJECTIVE:

To optimize the display interface design in complex flight tasks, the present study aimed to propose a dynamic conceptual framework and a timeline task analysis method for the quantization of the dynamic time effect of mental workload and the influencing factors of task types in the mental workload prediction model.

METHODS:

The multi-factor mental workload prediction model based on attention resource allocation was integrated to establish the dynamic prediction model of mental workload. The ergonomics simulation experiment was carried out by recording the data on the performance of embedded subtasks, National Aeronautics and Space Administration-Task Load Index (NASA-TLX) subjective evaluation, and eye tracking.

RESULTS:

The results indicated that the prediction model had a good prediction accuracy and effectiveness under different simulated interfaces and complex tasks, and the real-time monitoring of pilots’ mental workload state was realized.

CONCLUSION:

In conclusion, the prediction model and the experimental method could be applied to avoid the overload of the pilot throughout the flight phase by optimizing the display interface and adjusting the flight task.

A Neuroergonomics Approach to Mental Workload, Engagement and Human Performance

The assessment and prediction of cognitive performance is a key issue for any discipline concerned with human operators in the context of safety-critical behavior. Most of the research has focused on the measurement of mental workload but this construct remains difficult to operationalize despite decades of research on the topic. Recent advances in Neuroergonomics have expanded our understanding of neurocognitive processes across different operational domains. We provide a framework to disentangle those neural mechanisms that underpin the relationship between task demand, arousal, mental workload and human performance. This approach advocates targeting those specific mental states that precede a reduction of performance efficacy. A number of undesirable neurocognitive states (mind wandering, effort withdrawal, perseveration, inattentional phenomena) are identified and mapped within a two-dimensional conceptual space encompassing task engagement and arousal. We argue that monitoring the prefrontal cortex and its deactivation can index a generic shift from a nominal operational state to an impaired one where performance is likely to degrade. Neurophysiological, physiological and behavioral markers that specifically account for these states are identified. We then propose a typology of neuroadaptive countermeasures to mitigate these undesirable mental states.

Understanding individualistic response patterns when assessing expert operators on nuclear power plant control tasks

We evaluated the performance of three highly practiced participants on three task types that comprised a simulated nuclear power plant control operation. Multiple subjective, physiological, and objective performance measures were collected on these three highly-practiced individuals. Results indicated ceiling effects in terms of performance accuracy, yet each individual adopted a unique response strategy across the respective sub-tasks. Their maximised accuracy was achieved at the expense of longer response times across differing sub-tasks. The measures which proved diagnostic and predictive of performance capacity were explored. The current conclusion presents us with an invidious problem in that performance and workload associations, insensitivities, and dissociations may be unique to each individual operator, and may well depend also upon the overall task in context. Such findings push our science away from seeking nomothetic assertions and toward individuated concerns. In consequence, the age of the idiographic may well be upon us. Practitioner summary: The importance and relevance of nuclear power control is self-evident. Concerns here have centred around the safety of the technology and its operators. Our work informs practitioners in this industry, and in Ergonomics in general, of the response of highly trained individuals in these safety-critical, operational domains. We show that even experts engage in personal and individual strategies, an observation critical to the assessment of this specific workplace, and potentially all others. Abbreviations: NPP: nuclear power plant; ROs: reactor operators; MCR: main control room; LOA: levels of automation; EOP: emergency operating procedure; OP: operating procedures; ISA: instantaneous self-assessment; DSSQ: Dundee stres state questionnaire.

Motor Performance, Mental Workload and Self-Efficacy Dynamics during Learning of Reaching Movements throughout Multiple Practice Sessions

The human capability to learn new motor skills depends on the efficient engagement of cognitive-motor resources, as reflected by mental workload, and psychological mechanisms (e.g., self-efficacy). While numerous investigations have examined the relationship between motor behavior and mental workload or self-efficacy in a performance context, a fairly limited effort focused on the combined examination of these notions during learning. Thus, this study aimed to examine their concomitant dynamics during the learning of a novel reaching skill practiced throughout multiple sessions. Individuals had to learn to control a virtual robotic arm via a human-machine interface by using limited head motion throughout eight practice sessions while motor performance, mental workload, and self-efficacy were assessed. The results revealed that as individuals learned to control the robotic arm, performance improved at the fastest rate, followed by a more gradual reduction of mental workload and finally an increase in self-efficacy. These results suggest that once the performance improved, less cognitive-motor resources were recruited, leading to an attenuated mental workload. Considering that attention is a primary cognitive resource driving mental workload, it is suggested that during early learning, attentional resources are primarily allocated to address task demands and not enough are available to assess self-efficacy. However, as the performance becomes more automatic, a lower level of mental workload is attained driven by decreased recruitment of attentional resources. These available resources allow for a reliable assessment of self-efficacy resulting in a subsequent observable change. These results are also discussed in terms of the application to the training and design of assistive technologies.

Mental workload characteristics of manipulator teleoperators with different spatial cognitive abilities

The main research on manipulator teleoperation includes robust of high-degree of freedom manipulators, sensor measurement accuracy, time delay, and mechanical structure design. Increased mental capacity requirements for complex assignments result in an increased mental workload. Spatial cognitive ability was considered to be the key factor affecting teleoperation performance. To accomplish this, we had 50 participants performed teleoperation while recorded their electroencephalogram. Electroencephalogram data of each task were divided into two periods, which correspond to the observation and large-scale transfer stages of teleoperation, respectively (period 1) and adjust the attitude of the manipulator to approach and align with the target stage (period 2). Brain topographic maps of period 1 (period 1 wavelet packet energy minus resting state wavelet packet energy) and period 2 (period 2 wavelet packet energy minus resting state wavelet packet energy) show that the frontal, central, and occipital regions are the main working areas of low spatial cognitive operators in period 1, while the frontal, central, and occipital regions are the main working areas of high spatial cognitive operators in period 1. The main changes in period 2 were frontal, central, parietal, and occipital regions. This study has implication for the analysis of electroencephalogram signal characteristics of mental workload in different populations to improve operators’ well-being and safety at teleoperation work.

Aging and eye tracking: in the quest for objective biomarkers

Recent applications of eye tracking for diagnosis, prognosis and follow-up of therapy in age-related neurological or psychological deficits have been reviewed. The review is focused on active aging, neurodegeneration and cognitive impairments. The potential impacts and current limitations of using characterizing features of eye movements and pupillary responses (oculometrics) as objective biomarkers in the context of aging are discussed. A closer look into the findings, especially with respect to cognitive impairments, suggests that eye tracking is an invaluable technique to study hidden aspects of aging that have not been revealed using any other noninvasive tool. Future research should involve a wider variety of oculometrics, in addition to saccadic metrics and pupillary responses, including nonlinear and combinatorial features as well as blink- and fixation-related metrics to develop biomarkers to trace age-related irregularities associated with cognitive and neural deficits.

Cognitive Engineering to Improve Patient Safety and Outcomes in Cardiothoracic Surgery

Cognitive engineering is focused on how humans can cope and master the complexity of processes and technological environments. In cardiothoracic surgery, the goal is to support safe and effective human performance by preventing medical errors. Strategies derived from cognitive engineering research could be introduced in cardiothoracic surgery practice in the near future to enhance patient safety and outcomes.

Assessment of perceived workload in academic health center community pharmacies before and after implementation of a central call center

PURPOSE

Results of a study to determine whether reducing pharmacy phone call workload through implementation of a pharmacy services call center (PSCC) led to decreased employee workload, improved efficiency, and increased pharmacist availability for patient care are reported.

METHODS

A pre-post study was conducted using the NASA Task Load Index (NASA-TLX) instrument. Pharmacists, pharmacy technicians at 7 academic health center community pharmacies, and PSCC staff provided NASA-TLX data over 5 days during 3 data collection periods before and after PSCC implementation. Perceived workload was measured as an overall workload score (OWS) and mean scores for 6 NASA-TLX workload dimensions (mental demand, physical demand, temporal demand, performance, effort, and frustration).

RESULTS

Relative to pre-PSCC values, mean postimplementation OWS scores significantly decreased in all 7 pharmacies (from 33.3 to 29.1 overall, p < 0.001) but especially in small pharmacies (from 31.7 to 27.6, p < 0.001). Scores for the physical demand and frustration dimensions were low in both the PSCC and in the 7 pharmacies, while scores for the performance dimension remained high (range, 6.8-8.3). In general, scores for all other measured NASA-TLX dimensions decreased after PSCC implementation, more so at smaller pharmacies. The PSCC staff mean OWS score increased over time (from 26.8 to 28.6, p < 0.0001) but remained near the overall pharmacy average of 29.1.

CONCLUSION

Use of the NASA TLX allowed for a direct subjective measurement of workload as perceived by pharmacy and PSCC employees before and after PSCC implementation. Long-term effects of the PSCC on workload should be assessed.

Theories and Methods for Labeling Cognitive Workload: Classification and Transfer Learning

There are a number of key data-centric questions that must be answered when developing classifiers for operator functional states. "Should a supervised or unsupervised learning approach be used? What degree of labeling and transformation must be performed on the data? What are the trade-offs between algorithm flexibility and model interpretability, as generally these features are at odds?" Here, we focus exclusively on the labeling of cognitive load data for supervised learning. We explored three methods of labeling cognitive states for three-state classification. The first method labels states derived from a tertiary split of trial difficulty during a spatial memory task. The second method was more adaptive; it employed a mixed-effects stress-strain curve and estimated an individual's performance asymptotes with respect to the same spatial memory task. The final method was similar to the second approach; however, it employed a mixed-effects Rasch model to estimate individual capacity limits within the context of item response theory for the spatial memory task. To assess the strength of each of these labeling approaches, we compared the area under the curve (AUC) for receiver operating curves (ROCs) from elastic net and random forest classifiers. We chose these classifiers based on a combination of interpretability, flexibility, and past modeling success. We tested these techniques across two groups of individuals and two tasks to test the effects of different labeling techniques on cross-person and cross-task transfer. Overall, we observed that the Rasch model labeling paired with a random forest classifier led to the best model fits and showed evidence of both cross-person and cross-task transfer.

Human-based dynamics of mental workload in complicated systems

As a dynamic system in which different factors affect human performance via dynamic interactions, mental workload needs a dynamic measure to monitor its factors and evidence in a complicated system, an approach that is lacking in the literature. The present study introduces a system dynamics-based model for designing feedback mechanisms related to the mental workload through literature review and content analysis of the previous studies. A human-based archetype of mental workload was detected from the data collection process. The archetype is presented at various stages, including dynamic theory, behavior over time, leverage points and model verification. The real validation of the dynamic model was confirmed in an urban train simulator. The dynamic model can be used to analyze the long-term behavior of the mental workload. Decision-makers can benefit from the developed archetypes in evaluating the dynamic impact of their decisions on accident prevention in the complicated systems.

A Systematic Review of Physiological Measures of Mental Workload

Mental workload (MWL) can affect human performance and is considered critical in the design and evaluation of complex human-machine systems. While numerous physiological measures are used to assess MWL, there appears no consensus on their validity as effective agents of MWL. This study was conducted to provide a comprehensive understanding of the use of physiological measures of MWL and to synthesize empirical evidence on the validity of the measures to discriminate changes in MWL. A systematical literature search was conducted with four electronic databases for empirical studies measuring MWL with physiological measures. Ninety-one studies were included for analysis. We identified 78 physiological measures, which were distributed in cardiovascular, eye movement, electroencephalogram (EEG), respiration, electromyogram (EMG) and skin categories. Cardiovascular, eye movement and EEG measures were the most widely used across varied research domains, with 76%, 66%, and 71% of times reported a significant association with MWL, respectively. While most physiological measures were found to be able to discriminate changes in MWL, they were not universally valid in all task scenarios. The use of physiological measures and their validity for MWL assessment also varied across different research domains. Our study offers insights into the understanding and selection of appropriate physiological measures for MWL assessment in varied human-machine systems.

Macaques preferentially attend to visual patterns with higher fractal dimension contours

Animals' sensory systems evolved to efficiently process information from their environmental niches. Niches often include irregular shapes and rough textures (e.g., jagged terrain, canopy outlines) that must be navigated to find food, escape predators, and master other fitness-related challenges. For most primates, vision is the dominant sensory modality and thus, primates have evolved systems for processing complicated visual stimuli. One way to quantify information present in visual stimuli in natural scenes is evaluating their fractal dimension. We hypothesized that sensitivity to complicated geometric forms, indexed by fractal dimension, is an evolutionarily conserved capacity, and tested this capacity in rhesus macaques (Macaca mulatta). Monkeys viewed paired black and white images of simulated self-similar contours that systematically varied in fractal dimension while their attention to the stimuli was measured using noninvasive infrared eye tracking. They fixated more frequently on, dwelled for longer durations on, and had attentional biases towards images that contain boundary contours with higher fractal dimensions. This indicates that, like humans, they discriminate between visual stimuli on the basis of fractal dimension and may prefer viewing informationally rich visual stimuli. Our findings suggest that sensitivity to fractal dimension may be a wider ability of the vertebrate vision system.