Evaluation of PERCLOS based current fatigue monitoring technologies

Driving Fatigue Detection Based on Hybrid Electroencephalography and Eye Tracking

EEG-based unimodal method has demonstrated significant success in the detection of driving fatigue. Nonetheless, data from a single modality might be not sufficient to optimize fatigue detection due to incomplete information. To address this limitation and enhance the performance of driving fatigue detection, a novel multimodal architecture combining hybrid electroencephalograph (EEG) and eye tracking data was proposed in this work. Specifically, the EEG and eye tracking data were separately input into encoders, generating two one-dimensional (1D) features. Subsequently, these 1D features were fed into a cross-modal predictive alignment module to improve fusion efficiency and two 1D attention modules to enhance feature representation. Furthermore, the fused features were recognized by a linear classifier. To evaluate the effectiveness of the proposed multimodal method, comprehensive validation tasks were conducted, including intra-session, cross-session, and cross-subject evaluations. In the intra-session task, the proposed architecture achieves an exceptional average accuracy of 99.93%. Moreover, in the cross-session task, our method demonstrates an average accuracy of 88.67%, surpassing the performance of EEG-only approach by 8.52%, eye tracking-only method by 5.92%, multimodal deep canonical correlation analysis (DCCA) technique by 0.42%, and multimodal deep generalized canonical correlation analysis (DGCCA) approach by 0.84%. Similarly, in the cross-subject task, the proposed approach achieves an average accuracy of 78.19%, outperforming EEG-only method by 5.87%, eye tracking-only approach by 4.21%, DCCA method by 0.55%, and DGCCA approach by 0.44%. The experimental results conclusively illustrate the superior effectiveness of the proposed method compared to both single modality approaches and canonical correlation analysis-based multimodal methods.

Predicting Drowsy Driver Break Taking During Long Drives

OBJECTIVE

The current study investigated the factors that predict drowsy drivers' decisions regarding whether to take breaks versus continue driving during long simulator drives.

BACKGROUND

Driver drowsiness contributes to substantial numbers of motor vehicle crashes, injuries, and deaths. Previous research has shown that taking a nap and consuming caffeine can temporarily mitigate drowsiness and enable continued safe driving.

METHOD

Ninety drivers completed a 150-mile highway drive in a driving simulator after a day of partial sleep restriction. Drivers passed several simulated rest areas where they could take breaks. To replicate drivers' motivation to reach their destination safely but also quickly, drivers were told that they would be paid more for completing the simulated drive faster but would forfeit their payment if they crashed.

RESULTS

Break taking was predicted by drivers' self-ratings of drowsiness and by the severity of lane departures. However, even at the highest levels of drowsiness, most drivers bypassed simulated rest areas without stopping. In comparing self-rated drowsiness to drowsiness measured by eye closures, drivers often under- and over-estimate their own level of drowsiness.

CONCLUSION

Drowsy drivers use their own self-assessed drowsiness when deciding whether to take breaks. These self-assessments are often incorrect, and even when drivers rate themselves as severely drowsy they are unlikely to stop to rest during long drives.

APPLICATION

The findings reveal the need for effective drowsy driving countermeasures to motivate drivers to stop to take breaks. Results underscore the need to educate and/or motivate drivers to respond sooner to warning signs of drowsiness.

A dataset for fatigue estimation during shoulder internal and external rotation movements using wearables

Wearable sensors have recently been extensively used in sports science, physical rehabilitation, and industry providing feedback on physical fatigue. Information obtained from wearable sensors can be analyzed by predictive analytics methods, such as machine learning algorithms, to determine fatigue during shoulder joint movements, which have complex biomechanics. The presented dataset aims to provide data collected via wearable sensors during a fatigue protocol involving dynamic shoulder internal rotation (IR) and external rotation (ER) movements. Thirty-four healthy subjects performed shoulder IR and ER movements with different percentages of maximal voluntary isometric contraction (MVIC) force until they reached the maximal exertion. The dataset includes demographic information, anthropometric measurements, MVIC force measurements, and digital data captured via surface electromyography, inertial measurement unit, and photoplethysmography, as well as self-reported assessments using the Borg rating scale of perceived exertion and the Karolinska sleepiness scale. This comprehensive dataset provides valuable insights into physical fatigue assessment, allowing the development of fatigue detection/prediction algorithms and the study of human biomechanical characteristics during shoulder movements within a fatigue protocol.

Psychophysiological models of hypovigilance detection: A scoping review

Hypovigilance represents a major contributor to accidents. In operational contexts, the burden of monitoring/managing vigilance often rests on operators. Recent advances in sensing technologies allow for the development of psychophysiology-based (hypo)vigilance prediction models. Still, these models remain scarcely applied to operational situations and need better understanding. The current scoping review provides a state of knowledge regarding psychophysiological models of hypovigilance detection. Records evaluating vigilance measuring tools with gold standard comparisons and hypovigilance prediction performances were extracted from MEDLINE, PsychInfo, and Inspec. Exclusion criteria comprised aspects related to language, non-empirical papers, and sleep studies. The Quality Assessment tool for Diagnostic Accuracy Studies (QUADAS) and the Prediction model Risk Of Bias ASsessment Tool (PROBAST) were used for bias evaluation. Twenty-one records were reviewed. They were mainly characterized by participant selection and analysis biases. Papers predominantly focused on driving and employed several common psychophysiological techniques. Yet, prediction methods and gold standards varied widely. Overall, we outline the main strategies used to assess hypovigilance, their principal limitations, and we discuss applications of these models.

Adapting shift work schedules for sleep quality, sleep duration, and sleepiness in shift workers

BACKGROUND

Shift work is associated with insufficient sleep, which can compromise worker alertness with ultimate effects on occupational health and safety. Adapting shift work schedules may reduce adverse occupational outcomes.

OBJECTIVES

To assess the effects of shift schedule adaptation on sleep quality, sleep duration, and sleepiness among shift workers.

SEARCH METHODS

We searched CENTRAL, PubMed, Embase, and eight other databases on 13 December 2020, and again on 20 April 2022, applying no language restrictions.

SELECTION CRITERIA

We included randomised controlled trials (RCTs) and non-RCTs, including controlled before-after (CBA) trials, interrupted time series, and cross-over trials. Eligible trials evaluated any of the following shift schedule components. • Permanency of shifts • Regularity of shift changes • Direction of shift rotation • Speed of rotation • Shift duration • Timing of start of shifts • Distribution of shift schedule • Time off between shifts • Split shifts • Protected sleep • Worker participation We included studies that assessed sleep quality off-shift, sleep duration off-shift, or sleepiness during shifts.

DATA COLLECTION AND ANALYSIS

Two review authors independently screened the titles and abstracts of the records recovered by the search, read through the full-text articles of potentially eligible studies, and extracted data. We assessed the risk of bias of included studies using the Cochrane risk of bias tool, with specific additional domains for non-randomised and cluster-randomised studies. For all stages, we resolved any disagreements by consulting a third review author. We presented the results by study design and combined clinically homogeneous studies in meta-analyses using random-effects models. We assessed the certainty of the evidence with GRADE.

MAIN RESULTS

We included 11 studies with a total of 2125 participants. One study was conducted in a laboratory setting and was not considered for drawing conclusions on intervention effects. The included studies investigated different and often multiple changes to shift schedule, and were heterogeneous with respect to outcome measurement. Forward versus backward rotation Three CBA trials (561 participants) investigated the effects of forward rotation versus backward rotation. Only one CBA trial provided sufficient data for the quantitative analysis; it provided very low-certainty evidence that forward rotation compared with backward rotation did not affect sleep quality measured with the Basic Nordic Sleep Questionnaire (BNSQ; mean difference (MD) -0.20 points, 95% confidence interval (CI) -2.28 to 1.89; 62 participants) or sleep duration off-shift (MD -0.21 hours, 95% CI -3.29 to 2.88; 62 participants). However, there was also very low-certainty evidence that forward rotation reduced sleepiness during shifts measured with the BNSQ (MD -1.24 points, 95% CI -2.24 to -0.24; 62 participants). Faster versus slower rotation Two CBA trials and one non-randomised cross-over trial (341 participants) evaluated faster versus slower shift rotation. We were able to meta-analyse data from two studies. There was low-certainty evidence of no difference in sleep quality off-shift (standardised mean difference (SMD) -0.01, 95% CI -0.26 to 0.23) and very low-certainty evidence that faster shift rotation reduced sleep duration off-shift (SMD -0.26, 95% CI -0.51 to -0.01; 2 studies, 282 participants). The SMD for sleep duration translated to an MD of 0.38 hours' less sleep per day (95% CI -0.74 to -0.01). One study provided very low-certainty evidence that faster rotations decreased sleepiness during shifts measured with the BNSQ (MD -1.24 points, 95% CI -2.24 to -0.24; 62 participants). Limited shift duration (16 hours) versus unlimited shift duration Two RCTs (760 participants) evaluated 80-hour workweeks with maximum daily shift duration of 16 hours versus workweeks without any daily shift duration limits. There was low-certainty evidence that the 16-hour limit increased sleep duration off-shift (SMD 0.50, 95% CI 0.21 to 0.78; which translated to an MD of 0.73 hours' more sleep per day, 95% CI 0.30 to 1.13; 2 RCTs, 760 participants) and moderate-certainty evidence that the 16-hour limit reduced sleepiness during shifts, measured with the Karolinska Sleepiness Scale (SMD -0.29, 95% CI -0.44 to -0.14; which translated to an MD of 0.37 fewer points, 95% CI -0.55 to -0.17; 2 RCTs, 716 participants). Shorter versus longer shifts One RCT, one CBA trial, and one non-randomised cross-over trial (692 participants) evaluated shorter shift duration (eight to 10 hours) versus longer shift duration (two to three hours longer). There was very low-certainty evidence of no difference in sleep quality (SMD -0.23, 95% CI -0.61 to 0.15; which translated to an MD of 0.13 points lower on a scale of 1 to 5; 2 studies, 111 participants) or sleep duration off-shift (SMD 0.18, 95% CI -0.17 to 0.54; which translated to an MD of 0.26 hours' less sleep per day; 2 studies, 121 participants). The RCT and the non-randomised cross-over study found that shorter shifts reduced sleepiness during shifts, while the CBA study found no effect on sleepiness. More compressed versus more spread out shift schedules One RCT and one CBA trial (346 participants) evaluated more compressed versus more spread out shift schedules. The CBA trial provided very low-certainty evidence of no difference between the groups in sleep quality off-shift (MD 0.31 points, 95% CI -0.53 to 1.15) and sleep duration off-shift (MD 0.52 hours, 95% CI -0.52 to 1.56).

AUTHORS' CONCLUSIONS

Forward and faster rotation may reduce sleepiness during shifts, and may make no difference to sleep quality, but the evidence is very uncertain. Very low-certainty evidence indicated that sleep duration off-shift decreases with faster rotation. Low-certainty evidence indicated that on-duty workweeks with shift duration limited to 16 hours increases sleep duration, with moderate-certainty evidence for minimal reductions in sleepiness. Changes in shift duration and compression of workweeks had no effect on sleep or sleepiness, but the evidence was of very low-certainty. No evidence is available for other shift schedule changes. There is a need for more high-quality studies (preferably RCTs) for all shift schedule interventions to draw conclusions on the effects of shift schedule adaptations on sleep and sleepiness in shift workers.

A review of cyber vigilance tasks for network defense

The capacity to sustain attention to virtual threat landscapes has led cyber security to emerge as a new and novel domain for vigilance research. However, unlike classic domains, such as driving and air traffic control and baggage security, very few vigilance tasks exist for the cyber security domain. Four essential challenges that must be overcome in the development of a modern, validated cyber vigilance task are extracted from this review of existent platforms that can be found in the literature. Firstly, it can be difficult for researchers to access confidential cyber security systems and personnel. Secondly, network defense is vastly more complex and difficult to emulate than classic vigilance domains such as driving. Thirdly, there exists no single, common software console in cyber security that a cyber vigilance task could be based on. Finally, the rapid pace of technological evolution in network defense correspondingly means that cyber vigilance tasks can become obsolete just as quickly. Understanding these challenges is imperative in advancing human factors research in cyber security.

CCS categories

Human-centered computing~Human computer interaction (HCI)~HCI design and evaluation methods.

A CNN-Based Wearable System for Driver Drowsiness Detection

Drowsiness poses a serious challenge to road safety and various in-cabin sensing technologies have been experimented with to monitor driver alertness. Cameras offer a convenient means for contactless sensing, but they may violate user privacy and require complex algorithms to accommodate user (e.g., sunglasses) and environmental (e.g., lighting conditions) constraints. This paper presents a lightweight convolution neural network that measures eye closure based on eye images captured by a wearable glass prototype, which features a hot mirror-based design that allows the camera to be installed on the glass temples. The experimental results showed that the wearable glass prototype, with the neural network in its core, was highly effective in detecting eye blinks. The blink rate derived from the glass output was highly consistent with an industrial gold standard EyeLink eye-tracker. As eye blink characteristics are sensitive measures of driver drowsiness, the glass prototype and the lightweight neural network presented in this paper would provide a computationally efficient yet viable solution for real-world applications.

Detecting driver fatigue using heart rate variability: A systematic review

Driver fatigue detection systems have potential to improve road safety by preventing crashes and saving lives. Conventional driver monitoring systems based on driving performance and facial features may be challenged by the application of automated driving systems. This limitation could potentially be overcome by monitoring systems based on physiological measurements. Heart rate variability (HRV) is a physiological marker of interest for detecting driver fatigue that can be measured during real life driving. This systematic review investigates the relationship between HRV measures and driver fatigue, as well as the performance of HRV based fatigue detection systems. With the applied eligibility criteria, 18 articles were identified in this review. Inconsistent results can be found within the studies that investigated differences of HRV measures between alert and fatigued drivers. For studies that developed HRV based fatigue detection systems, the detection performance showed a large variation, where the detection accuracy ranged from 44% to 100%. The inconsistency and variation of the results can be caused by differences in several key aspects in the study designs. Progress in this field is needed to determine the relationship between HRV and different fatigue causal factors and its connection to driver performance. To be deployed, HRV-based fatigue detection systems need to be thoroughly tested in real life conditions with good coverage of relevant driving scenarios and a sufficient number of participants.

Automatic Detection of Driver Fatigue Based on EEG Signals Using a Developed Deep Neural Network

In recent years, detecting driver fatigue has been a significant practical necessity and issue. Even though several investigations have been undertaken to examine driver fatigue, there are relatively few standard datasets on identifying driver fatigue. For earlier investigations, conventional methods relying on manual characteristics were utilized to assess driver fatigue. In any case study, such approaches need previous information for feature extraction, which could raise computing complexity. The current work proposes a driver fatigue detection system, which is a fundamental necessity to minimize road accidents. Data from 11 people are gathered for this purpose, resulting in a comprehensive dataset. The dataset is prepared in accordance with previously published criteria. A deep convolutional neural network–long short-time memory (CNN–LSTM) network is conceived and evolved to extract characteristics from raw EEG data corresponding to the six active areas A, B, C, D, E (based on a single channel), and F. The study’s findings reveal that the suggested deep CNN–LSTM network could learn features hierarchically from raw EEG data and attain a greater precision rate than previous comparative approaches for two-stage driver fatigue categorization. The suggested approach may be utilized to construct automatic fatigue detection systems because of their precision and high speed.

Effects of Automation and Fatigue on Drivers from Various Age Groups

This study explores how drivers are affected by automation when driving in rested and fatigued conditions. Eighty-nine drivers (45 females, 44 males) aged between 20 and 85 years attended driving experiments on separate days, once in a rested and once in a fatigued condition, in a counterbalanced order. The results show an overall effect of automation to significantly reduce drivers’ workload and effort. The automation had different effects, depending on the drivers’ conditions. Differences between the manual and automated mode were larger for the perceived time pressure and effort in the fatigued condition as compared to the rested condition. Frustration was higher during manual driving when fatigued, but also higher during automated driving when rested. Subjective fatigue and the percentage of eye closure (PERCLOS) were higher in the automated mode compared to manual driving mode. PERCLOS differences between the automated and manual mode were higher in the fatigued condition than in the rested condition. There was a significant interaction effect of age and automation on drivers’ PERCLOS. These results are important for the development of driver-centered automation because they show different benefits for drivers of different ages, depending on their condition (fatigued or rested).

Driver drowsiness estimation using EEG signals with a dynamical encoder-decoder modeling framework

Drowsiness is a leading cause of accidents on the road as it negatively affects the driver’s ability to safely operate a vehicle. Neural activity recorded by EEG electrodes is a widely used physiological correlate of driver drowsiness. This paper presents a novel dynamical modeling solution to estimate the instantaneous level of the driver drowsiness using EEG signals, where the PERcentage of eyelid CLOSure (PERCLOS) is employed as the ground truth of driver drowsiness. Applying our proposed modeling framework, we find neural features present in EEG data that encode PERCLOS. In the decoding phase, we use a Bayesian filtering solution to estimate the PERCLOS level over time. A data set that comprises 18 driving tests, conducted by 13 drivers, has been used to investigate the performance of the proposed framework. The modeling performance in estimation of PERCLOS provides robust and repeatable results in tests with manual and automated driving modes by an average RMSE of 0.117 (at a PERCLOS range of 0 to 1) and average High Probability Density percentage of 62.5%. We further hypothesized that there are biomarkers that encode the PERCLOS across different driving tests and participants. Using this solution, we identified possible biomarkers such as Theta and Delta powers. Results show that about 73% and 66% of the Theta and Delta powers which are selected as biomarkers are increasing as PERCLOS grows during the driving test. We argue that the proposed method is a robust and reliable solution to estimate drowsiness in real-time which opens the door in utilizing EEG-based measures in driver drowsiness detection systems.

Electroencephalogram-Based Approaches for Driver Drowsiness Detection and Management: A Review

Drowsiness is not only a core challenge to safe driving in traditional driving conditions but also a serious obstacle for the wide acceptance of added services of self-driving cars (because drowsiness is, in fact, one of the most representative early-stage symptoms of self-driving carsickness). In view of the importance of detecting drivers’ drowsiness, this paper reviews the algorithms of electroencephalogram (EEG)-based drivers’ drowsiness detection (DDD). To facilitate the review, the EEG-based DDD approaches are organized into a tree structure taxonomy, having two main categories, namely “detection only (open-loop)” and “management (closed-loop)”, both aimed at designing better DDD systems that ensure early detection, reliability and practical utility. To achieve this goal, we addressed seven questions, the answers of which helped in developing an EEG-based DDD system that is superior to the existing ones. A basic assumption in this review article is that although driver drowsiness and carsickness-induced drowsiness are caused by different factors, the brain network that regulates drowsiness is the same.

Driver Drowsiness Detection by Applying Deep Learning Techniques to Sequences of Images

This work presents the development of an ADAS (advanced driving assistance system) focused on driver drowsiness detection, whose objective is to alert drivers of their drowsy state to avoid road traffic accidents. In a driving environment, it is necessary that fatigue detection is performed in a non-intrusive way, and that the driver is not bothered with alarms when he or she is not drowsy. Our approach to this open problem uses sequences of images that are 60 s long and are recorded in such a way that the subject’s face is visible. To detect whether the driver shows symptoms of drowsiness or not, two alternative solutions are developed, focusing on the minimization of false positives. The first alternative uses a recurrent and convolutional neural network, while the second one uses deep learning techniques to extract numeric features from images, which are introduced into a fuzzy logic-based system afterwards. The accuracy obtained by both systems is similar: around 65% accuracy over training data, and 60% accuracy on test data. However, the fuzzy logic-based system stands out because it avoids raising false alarms and reaches a specificity (proportion of videos in which the driver is not drowsy that are correctly classified) of 93%. Although the obtained results do not achieve very satisfactory rates, the proposals presented in this work are promising and can be considered a solid baseline for future works.

European NCAP Program Developments to Address Driver Distraction, Drowsiness and Sudden Sickness

Driver distraction and drowsiness remain significant contributors to death and serious injury on our roads and are long standing issues in road safety strategies around the world. With developments in automotive technology, including driver monitoring, there are now more options available for automotive manufactures to mitigate risks associated with driver state. Such developments in Occupant Status Monitoring (OSM) are being incorporated into the European New Car Assessment Programme (Euro NCAP) Safety Assist protocols. The requirements for OSM technologies are discussed along two dimensions: detection difficulty and behavioral complexity. More capable solutions will be able to provide higher levels of system availability, being the proportion of time a system could provide protection to the driver, and will be able to capture a greater proportion of complex real-word driver behavior. The testing approach could initially propose testing using both a dossier of evidence provided by the Original Equipment Manufacturer (OEM) alongside selected use of track testing. More capable systems will not rely only on warning strategies but will also include intervention strategies when a driver is not attentive. The roadmap for future OSM protocol development could consider a range of known and emerging safety risks including driving while intoxicated by alcohol or drugs, cognitive distraction, and the driver engagement requirements for supervision and take-over performance with assisted and automated driving features.

Evaluation of a Fatigue Detector Using Eye Closure-Associated Indicators Acquired from Truck Drivers in a Simulator Study

This paper presents a camera-based prototype sensor for detecting fatigue and drowsiness in drivers, which are common causes of road accidents. The evaluation of the detector operation involved eight professional truck drivers, who drove the truck simulator twice—i.e., when they were rested and drowsy. The Fatigue Symptoms Scales (FSS) questionnaire was used to assess subjectively perceived levels of fatigue, whereas the percentage of eye closure time (PERCLOS), eye closure duration (ECD), and frequency of eye closure (FEC) were selected as eye closure-associated fatigue indicators, determined from the images of drivers’ faces captured by the sensor. Three alternative models for subjective fatigue were used to analyse the relationship between the raw score of the FSS questionnaire, and the eye closure-associated indicators were estimated. The results revealed that, in relation to the subjective assessment of fatigue, PERCLOS is a significant predictor of the changes observed in individual subjects during the performance of tasks, while ECD reflects the individual differences in subjective fatigue occurred both between drivers and in individual drivers between the ‘rested’ and ‘drowsy’ experimental conditions well. No relationship between the FEC index and the FSS state scale was found.

Electro-Encephalography and Electro-Oculography in Aeronautics: A Review Over the Last Decade (2010-2020)

Electro-encephalography (EEG) and electro-oculography (EOG) are methods of electrophysiological monitoring that have potentially fruitful applications in neuroscience, clinical exploration, the aeronautical industry, and other sectors. These methods are often the most straightforward way of evaluating brain oscillations and eye movements, as they use standard laboratory or mobile techniques. This review describes the potential of EEG and EOG systems and the application of these methods in aeronautics. For example, EEG and EOG signals can be used to design brain-computer interfaces (BCI) and to interpret brain activity, such as monitoring the mental state of a pilot in determining their workload. The main objectives of this review are to, (i) offer an in-depth review of literature on the basics of EEG and EOG and their application in aeronautics; (ii) to explore the methodology and trends of research in combined EEG-EOG studies over the last decade; and (iii) to provide methodological guidelines for beginners and experts when applying these methods in environments outside the laboratory, with a particular focus on human factors and aeronautics. The study used databases from scientific, clinical, and neural engineering fields. The review first introduces the characteristics and the application of both EEG and EOG in aeronautics, undertaking a large review of relevant literature, from early to more recent studies. We then built a novel taxonomy model that includes 150 combined EEG-EOG papers published in peer-reviewed scientific journals and conferences from January 2010 to March 2020. Several data elements were reviewed for each study (e.g., pre-processing, extracted features and performance metrics), which were then examined to uncover trends in aeronautics and summarize interesting methods from this important body of literature. Finally, the review considers the advantages and limitations of these methods as well as future challenges.

Eye-Tracking Metrics Predict Perceived Workload in Robotic Surgical Skills Training

OBJECTIVE

The aim of this study is to assess the relationship between eye-tracking measures and perceived workload in robotic surgical tasks.

BACKGROUND

Robotic techniques provide improved dexterity, stereoscopic vision, and ergonomic control system over laparoscopic surgery, but the complexity of the interfaces and operations may pose new challenges to surgeons and compromise patient safety. Limited studies have objectively quantified workload and its impact on performance in robotic surgery. Although not yet implemented in robotic surgery, minimally intrusive and continuous eye-tracking metrics have been shown to be sensitive to changes in workload in other domains.

METHODS

Eight surgical trainees participated in 15 robotic skills simulation sessions. In each session, participants performed up to 12 simulated exercises. Correlation and mixed-effects analyses were conducted to explore the relationships between eye-tracking metrics and perceived workload. Machine learning classifiers were used to determine the sensitivity of differentiating between low and high workload with eye-tracking features.

RESULTS

Gaze entropy increased as perceived workload increased, with a correlation of .51. Pupil diameter and gaze entropy distinguished differences in workload between task difficulty levels, and both metrics increased as task level difficulty increased. The classification model using eye-tracking features achieved an accuracy of 84.7% in predicting workload levels.

CONCLUSION

Eye-tracking measures can detect perceived workload during robotic tasks. They can potentially be used to identify task contributors to high workload and provide measures for robotic surgery training.

APPLICATION

Workload assessment can be used for real-time monitoring of workload in robotic surgical training and provide assessments for performance and learning.

The Role of Eye Tracking Technology in Assessing Older Driver Safety

A growing body of literature is focused on the use of eye tracking (ET) technology to understand the association between objective visual parameters and higher order brain processes such as cognition. One of the settings where this principle has found practical utility is in the area of driving safety.

METHODS

We reviewed the literature to identify the changes in ET parameters with older adults and neurodegenerative disease.

RESULTS

This narrative review provides a brief overview of oculomotor system anatomy and physiology, defines common eye movements and tracking variables that are typically studied, explains the most common methods of eye tracking measurements during driving in simulation and in naturalistic settings, and examines the association of impairment in ET parameters with advanced age and neurodegenerative disease.

CONCLUSION

ET technology is becoming less expensive, more portable, easier to use, and readily applicable in a variety of clinical settings. Older adults and especially those with neurodegenerative disease may have impairments in visual search parameters, placing them at risk for motor vehicle crashes. Advanced driver assessment systems are becoming more ubiquitous in newer cars and may significantly reduce crashes related to impaired visual search, distraction, and/or fatigue.

A pre-drive ocular assessment predicts alertness and driving impairment: A naturalistic driving study in shift workers

Sleepiness is a major contributor to motor vehicle crashes and shift workers are particularly vulnerable. There is currently no validated objective field-based measure of sleep-related impairment prior to driving. Ocular parameters are promising markers of continuous driver alertness in laboratory and track studies, however their ability to determine fitness-to-drive in naturalistic driving is unknown. This study assessed the efficacy of a pre-drive ocular assessment for predicting sleep-related impairment in naturalistic driving, in rotating shift workers. Fifteen healthcare workers drove an instrumented vehicle for 2 weeks, while working a combination of day, evening and night shifts. The vehicle monitored lane departures and behavioural microsleeps (blinks >500 ms) during the drive. Immediately prior to driving, ocular parameters were assessed with a 4-min test. Lane departures and behavioural microsleeps occurred on 17.5 % and 10 % of drives that had pre-drive assessments, respectively. Pre-drive blink duration significantly predicted behavioural microsleeps and showed promise for predicting lane departures (AUC = 0.79 and 0.74). Pre-drive percentage of time with eyes closed had high accuracy for predicting lane departures and behavioural microsleeps (AUC = 0.73 and 0.96), although was not statistically significant. Pre-drive psychomotor vigilance task variables were not statistically significant predictors of lane departures. Self-reported sleep-related and hazardous driving events were significantly predicted by mean blink duration (AUC = 0.65 and 0.69). Measurement of ocular parameters pre-drive predict drowsy driving during naturalistic driving, demonstrating potential for fitness-to-drive assessment in operational environments.

Sigmoid Wake Probability Model for High-Resolution Detection of Drowsiness Using Electroencephalogram

An efficient and reliable method to detect drowsiness can reduce accidents and injuries related to drowsy driving. However, existing systems for detecting drowsiness are often of low-resolution, expensive, and dependent on external parameters. Therefore, the goal of this study is to develop a high-resolution and efficient drowsiness detection algorithm using relatively less noisy sleep study data. To this end, we recorded electroencephalogram (EEG) from 53 subjects during a sleep study and leveraged the EEG frequency band changes at sleep onset to develop a model for drowsiness detection. The model devised herein provided a likelihood of wakefulness for 3-s signal segments. By choosing appropriate thresholds of the model output, we have identified three clusters that represent wakefulness, drowsiness, and, sleep. The proposed scheme has been validated using arousals which are cases of alertness and deep sleep segments, cluster quality evaluation metrics, graphical, and statistical analyses. The results presented in this work suggest that spectral properties of EEG can be utilized for high-resolution drowsiness detection in sleep study. Upon its successful validation in a driving study, the proposed model can lead to the development of an efficient drowsy driving monitoring system.

Sensor Networks for Aerospace Human-Machine Systems

Intelligent automation and trusted autonomy are being introduced in aerospace cyber-physical systems to support diverse tasks including data processing, decision-making, information sharing and mission execution. Due to the increasing level of integration/collaboration between humans and automation in these tasks, the operational performance of closed-loop human-machine systems can be enhanced when the machine monitors the operator's cognitive states and adapts to them in order to maximise the effectiveness of the Human-Machine Interfaces and Interactions (HMI²). Technological developments have led to neurophysiological observations becoming a reliable methodology to evaluate the human operator's states using a variety of wearable and remote sensors. The adoption of sensor networks can be seen as an evolution of this approach, as there are notable advantages if these sensors collect and exchange data in real-time, while their operation is controlled remotely and synchronised. This paper discusses recent advances in sensor networks for aerospace cyber-physical systems, focusing on Cognitive HMI² (CHMI²) implementations. The key neurophysiological measurements used in this context and their relationship with the operator's cognitive states are discussed. Suitable data analysis techniques based on machine learning and statistical inference are also presented, as these techniques allow processing both neurophysiological and operational data to obtain accurate cognitive state estimations. Lastly, to support the development of sensor networks for CHMI² applications, the paper addresses the performance characterisation of various state-of-the-art sensors and the propagation of measurement uncertainties through a machine learning-based inference engine. Results show that a proper sensor selection and integration can support the implementation of effective human-machine systems for various challenging aerospace applications, including Air Traffic Management (ATM), commercial airliner Single-Pilot Operations (SIPO), one-to-many Unmanned Aircraft Systems (UAS), and space operations management.

Inside the clinical evaluation of sleepiness: subjective and objective tools

PURPOSE

To critically review the available tools for evaluating excessive daytime sleepiness (EDS) in clinical practice.

METHODS

Objective tests and subjective scales were divided into three groups in accordance with the different dimensions of sleepiness they measure, namely physiological, manifest, and introspective. Strengths, weaknesses, and limitations of each test have been analysed and discussed along with the available recommendations for their use in clinical practice.

RESULTS

The majority of the tests developed for sleepiness evaluation do not have practical usefulness outside the research setting. The suboptimal correlation between different tests mainly depends on the different dimensions of sleepiness they analyse. Most importantly in-laboratory tests poorly correlate with sleepiness in real-life situations and, to date, none is able to predict the risk of injuries related to EDS, especially on an individual level.

CONCLUSIONS

There exists not the one best test to assess EDS, however, clinicians can choose a more specific test to address a specific diagnostic challenge on the individual level. The development of novel performance tests with low cost and easy to administer is advisable for both screening purposes and fitness for duty evaluations in populations at high risk of EDS-related injuries, for example professional drivers.

A prospective study on peptide mapping of human fatigue saliva markers based on magnetic beads

In order to explore convenient and stable fatigue markers, we studied various high-molecular-weight peptide fragments under fatigue state and non-fatigue state in the saliva using time of flight mass spectrometry. The saliva samples were collected from 10 healthy volunteers that were in the condition of fatigue and non-fatigue, respectively. Moreover, the time of flight mass spectrometry was conducted using two kinds of sample treatment methods, the magnetic beads enrichment (MB) and direct detection of stock solution. This was followed by modeling via the mass spectra of MB and supernatant (stock solution) directly collected after centrifugation. Both MB and direct sampling produced good spectrograms between 1,000 and 15,000 Da, while some peaks were lost in the enrichment. The spectrograms in the early and late period were different in each individual. Due to the limited sample size, 20 early and 20 late spectrograms were used for modeling analysis. Three different peptides were identified in the stock solution samples that can be detected in both fatigue and non-fatigue groups. The cross validity of MB model was 92.06%, while that of the stock solution model was 95.49%. The results showed that there were different peaks within the molecular weight of 2,000-15,000 Da, which provided a scientific basis for further realization of the convenient fatigue detection method based on the biosensor technique, with important theoretical and practical significance.

Person-directed, non-pharmacological interventions for sleepiness at work and sleep disturbances caused by shift work

BACKGROUND

Shift work is often associated with sleepiness and sleep disorders. Person-directed, non-pharmacological interventions may positively influence the impact of shift work on sleep, thereby improving workers' well-being, safety, and health.

OBJECTIVES

To assess the effects of person-directed, non-pharmacological interventions for reducing sleepiness at work and improving the length and quality of sleep between shifts for shift workers.

SEARCH METHODS

We searched CENTRAL, MEDLINE Ovid, Embase, Web of Knowledge, ProQuest, PsycINFO, OpenGrey, and OSH-UPDATE from inception to August 2015. We also screened reference lists and conference proceedings and searched the World Health Organization (WHO) Trial register. We contacted experts to obtain unpublished data.

SELECTION CRITERIA

Randomised controlled trials (RCTs) (including cross-over designs) that investigated the effect of any person-directed, non-pharmacological intervention on sleepiness on-shift or sleep length and sleep quality off-shift in shift workers who also work nights.

DATA COLLECTION AND ANALYSIS

At least two authors screened titles and abstracts for relevant studies, extracted data, and assessed risk of bias. We contacted authors to obtain missing information. We conducted meta-analyses when pooling of studies was possible.

MAIN RESULTS

We included 17 relevant trials (with 556 review-relevant participants) which we categorised into three types of interventions: (1) various exposures to bright light (n = 10); (2) various opportunities for napping (n = 4); and (3) other interventions, such as physical exercise or sleep education (n = 3). In most instances, the studies were too heterogeneous to pool. Most of the comparisons yielded low to very low quality evidence. Only one comparison provided moderate quality evidence. Overall, the included studies' results were inconclusive. We present the results regarding sleepiness below. Bright light Combining two comparable studies (with 184 participants altogether) that investigated the effect of bright light during the night on sleepiness during a shift, revealed a mean reduction 0.83 score points of sleepiness (measured via the Stanford Sleepiness Scale (SSS) (95% confidence interval (CI) -1.3 to -0.36, very low quality evidence). Another trial did not find a significant difference in overall sleepiness on another sleepiness scale (16 participants, low quality evidence).Bright light during the night plus sunglasses at dawn did not significantly influence sleepiness compared to normal light (1 study, 17 participants, assessment via reaction time, very low quality evidence).Bright light during the day shift did not significantly reduce sleepiness during the day compared to normal light (1 trial, 61 participants, subjective assessment, low quality evidence) or compared to normal light plus placebo capsule (1 trial, 12 participants, assessment via reaction time, very low quality evidence). Napping during the night shiftA meta-analysis on a single nap opportunity and the effect on the mean reaction time as a surrogate for sleepiness, resulted in a 11.87 ms reduction (95% CI 31.94 to -8.2, very low quality evidence). Two other studies also reported statistically non-significant decreases in reaction time (1 study seven participants; 1 study 49 participants, very low quality evidence).A two-nap opportunity resulted in a statistically non-significant increase of sleepiness (subjective assessment) in one study (mean difference (MD) 2.32, 95% CI -24.74 to 29.38, 1 study, 15 participants, low quality evidence). Other interventionsPhysical exercise and sleep education interventions showed promise, but sufficient data to draw conclusions are lacking.

AUTHORS' CONCLUSIONS

Given the methodological diversity of the included studies, in terms of interventions, settings, and assessment tools, their limited reporting and the very low to low quality of the evidence they present, it is not possible to determine whether shift workers' sleepiness can be reduced or if their sleep length or quality can be improved with these interventions.We need better and adequately powered RCTs of the effect of bright light, and naps, either on their own or together and other non-pharmacological interventions that also consider shift workers' chronobiology on the investigated sleep parameters.

Estimating endogenous changes in task performance from EEG

Brain wave activity is known to correlate with decrements in behavioral performance as individuals enter states of fatigue, boredom, or low alertness.Many BCI technologies are adversely affected by these changes in user state, limiting their application and constraining their use to relatively short temporal epochs where behavioral performance is likely to be stable. Incorporating a passive BCI that detects when the user is performing poorly at a primary task, and adapts accordingly may prove to increase overall user performance. Here, we explore the potential for extending an established method to generate continuous estimates of behavioral performance from ongoing neural activity; evaluating the extended method by applying it to the original task domain, simulated driving; and generalizing the method by applying it to a BCI-relevant perceptual discrimination task. Specifically, we used EEG log power spectra and sequential forward floating selection (SFFS) to estimate endogenous changes in behavior in both a simulated driving task and a perceptual discrimination task. For the driving task the average correlation coefficient between the actual and estimated lane deviation was 0.37 ± 0.22 (μ ± σ). For the perceptual discrimination task we generated estimates of accuracy, reaction time, and button press duration for each participant. The correlation coefficients between the actual and estimated behavior were similar for these three metrics (accuracy = 0.25 ± 0.37, reaction time = 0.33 ± 0.23, button press duration = 0.36 ± 0.30). These findings illustrate the potential for modeling time-on-task decrements in performance from concurrent measures of neural activity.

The accuracy of eyelid movement parameters for drowsiness detection

STUDY OBJECTIVES

Drowsiness is a major risk factor for motor vehicle and occupational accidents. Real-time objective indicators of drowsiness could potentially identify drowsy individuals with the goal of intervening before an accident occurs. Several ocular measures are promising objective indicators of drowsiness; however, there is a lack of studies evaluating their accuracy for detecting behavioral impairment due to drowsiness in real time.

METHODS

In this study, eye movement parameters were measured during vigilance tasks following restricted sleep and in a rested state (n = 33 participants) at three testing points (n = 71 data points) to compare ocular measures to a gold standard measure of drowsiness (OSLER). The utility of these parameters for detecting drowsiness-related errors was evaluated using receiver operating characteristic curves (ROC) (adjusted by clustering for participant) and identification of optimal cutoff levels for identifying frequent drowsiness-related errors (4 missed signals in a minute using OSLER). Their accuracy was tested for detecting increasing frequencies of behavioral lapses on a different task (psychomotor vigilance task [PVT]).

RESULTS

Ocular variables which measured the average duration of eyelid closure (inter-event duration [IED]) and the ratio of the amplitude to velocity of eyelid closure were reliable indicators of frequent errors (area under the curve for ROC of 0.73 to 0.83, p < 0.05). IED produced a sensitivity and specificity of 71% and 88% for detecting ≥ 3 lapses (PVT) in a minute and 100% and 86% for ≥ 5 lapses. A composite measure of several eye movement characteristics (Johns Drowsiness Scale) provided sensitivities of 77% and 100% for detecting 3 and ≥ 5 lapses in a minute, with specificities of 85% and 83%, respectively.

CONCLUSIONS

Ocular measures, particularly those measuring the average duration of episodes of eye closure are promising real-time indicators of drowsiness.

Review of Techniques for Driver Fatigue Detection

Driver fatigue is a significant issue in driving safety. Large bodies of literature have examined fatigue detection methods and techniques. This paper provides a broad overview of the researches concerning related topics. So far, driver fatigue detection techniques developed can be divided into three categories, including techniques based on vehicle movement tracking, drivers behavioral features and electroencephalograph signals. In addition, a contrast between these techniques was presented and further researches were discussed.

A smartphone-based driver safety monitoring system using data fusion

This paper proposes a method for monitoring driver safety levels using a data fusion approach based on several discrete data types: eye features, bio-signal variation, in-vehicle temperature, and vehicle speed. The driver safety monitoring system was developed in practice in the form of an application for an Android-based smartphone device, where measuring safety-related data requires no extra monetary expenditure or equipment. Moreover, the system provides high resolution and flexibility. The safety monitoring process involves the fusion of attributes gathered from different sensors, including video, electrocardiography, photoplethysmography, temperature, and a three-axis accelerometer, that are assigned as input variables to an inference analysis framework. A Fuzzy Bayesian framework is designed to indicate the driver’s capability level and is updated continuously in real-time. The sensory data are transmitted via Bluetooth communication to the smartphone device. A fake incoming call warning service alerts the driver if his or her safety level is suspiciously compromised. Realistic testing of the system demonstrates the practical benefits of multiple features and their fusion in providing a more authentic and effective driver safety monitoring.