Machine Learning Models for the Classification of Sleep Deprivation Induced Performance Impairment During a Psychomotor Vigilance Task Using Indices of Eye and Face Tracking

High risk professions, such as pilots, police officers, and TSA agents, require sustained vigilance over long periods of time and/or under conditions of little sleep. This can lead to performance impairment in occupational tasks. Predicting impaired states before performance decrement manifests is critical to prevent costly and damaging mistakes. We hypothesize that machine learning models developed to analyze indices of eye and face tracking technologies can accurately predict impaired states. To test this we trained 12 types of machine learning algorithms using five methods of feature selection with indices of eye and face tracking to predict the performance of individual subjects during a psychomotor vigilance task completed at 2-h intervals during a 25-h sleep deprivation protocol. Our results show that (1) indices of eye and face tracking are sensitive to physiological and behavioral changes concomitant with impairment; (2) methods of feature selection heavily influence classification performance of machine learning algorithms; and (3) machine learning models using indices of eye and face tracking can correctly predict whether an individual's performance is “normal” or “impaired” with an accuracy up to 81.6%. These methods can be used to develop machine learning based systems intended to prevent operational mishaps due to sleep deprivation by predicting operator impairment, using indices of eye and face tracking.

Signal-to-Noise Ratio in PVT Performance as a Cognitive Measure of the Effect of Sleep Deprivation on the Fidelity of Information Processing

Study Objectives

There is a long-standing debate about the best way to characterize performance deficits on the psychomotor vigilance test (PVT), a widely used assay of cognitive impairment in human sleep deprivation studies. Here, we address this issue through the theoretical framework of the diffusion model and propose to express PVT performance in terms of signal-to-noise ratio (SNR).

Methods

From the equations of the diffusion model for one-choice, reaction-time tasks, we derived an expression for a novel SNR metric for PVT performance. We also showed that LSNR-a commonly used log-transformation of SNR-can be reasonably well approximated by a linear function of the mean response speed, LSNRapx. We computed SNR, LSNR, LSNRapx, and number of lapses for 1284 PVT sessions collected from 99 healthy young adults who participated in laboratory studies with 38 hr of total sleep deprivation.

Results

All four PVT metrics captured the effects of time awake and time of day on cognitive performance during sleep deprivation. The LSNR had the best psychometric properties, including high sensitivity, high stability, high degree of normality, absence of floor and ceiling effects, and no bias in the meaning of change scores related to absolute baseline performance.

Conclusions

The theoretical motivation of SNR and LSNR permits quantitative interpretation of PVT performance as an assay of the fidelity of information processing in cognition. Furthermore, with a conceptual and statistical meaning grounded in information theory and generalizable across scientific fields, LSNR in particular is a useful tool for systems-integrated fatigue risk management.

Classifying performance impairment in response to sleep loss using pattern recognition algorithms on single session testing

There is currently no "gold standard" marker of cognitive performance impairment resulting from sleep loss. We utilized pattern recognition algorithms to determine which features of data collected under controlled laboratory conditions could most reliably identify cognitive performance impairment in response to sleep loss using data from only one testing session, such as would occur in the "real world" or field conditions. A training set for testing the pattern recognition algorithms was developed using objective Psychomotor Vigilance Task (PVT) and subjective Karolinska Sleepiness Scale (KSS) data collected from laboratory studies during which subjects were sleep deprived for 26-52h. The algorithm was then tested in data from both laboratory and field experiments. The pattern recognition algorithm was able to identify performance impairment with a single testing session in individuals studied under laboratory conditions using PVT, KSS, length of time awake and time of day information with sensitivity and specificity as high as 82%. When this algorithm was tested on data collected under real-world conditions from individuals whose data were not in the training set, accuracy of predictions for individuals categorized with low performance impairment were as high as 98%. Predictions for medium and severe performance impairment were less accurate. We conclude that pattern recognition algorithms may be a promising method for identifying performance impairment in individuals using only current information about the individual's behavior. Single testing features (e.g., number of PVT lapses) with high correlation with performance impairment in the laboratory setting may not be the best indicators of performance impairment under real-world conditions. Pattern recognition algorithms should be further tested for their ability to be used in conjunction with other assessments of sleepiness in real-world conditions to quantify performance impairment in response to sleep loss.

A combined field and laboratory design for assessing the impact of night shift work on police officer operational performance

OBJECTIVES

This study assessed the utility of a combined field and laboratory research design for measuring the impact of consecutive night shift work on the sleepiness, vigilance, and driving performance of police patrol officers.

DESIGN

For police patrol officers working their normal night shift duty cycles, simulated driving performance and psychomotor vigilance were measured in a laboratory on two separate occasions: in the morning after the last of five consecutive 10.7-h night shifts, and at the same time in the morning after three consecutive days off duty. Order of participation in conditions was randomized among subjects.

SETTING

Subjects experienced manipulation of sleep schedules due to working night shifts in a real operational environment, but performance testing was conducted under controlled laboratory conditions.

PARTICIPANTS

N = 29 active-duty police patrol officers (27 male, 2 female; age 37.1 ± 6.3 years) working night shift schedules participated in this study.

RESULTS

Simulated driving performance, psychomotor vigilance, and subjective sleepiness were significantly degraded following 5 consecutive night shifts as compared to 3 consecutive days off duty, indicating that active-duty police officers are susceptible to performance degradation as a consequence of working nights.

CONCLUSIONS

This combined field and laboratory research design succeeded in bridging the gap between the realism of the operational environment and the control of laboratory performance testing, demonstrating that this is a useful approach for addressing the relationship between shift work induced fatigue and critical operational task performance.