Facial features and head movements obtained with a webcam correlate with performance deterioration during prolonged wakefulness

Vision-based estimation of fatigue and engagement in cognitive training sessions

Computerized cognitive training (CCT) is a scalable, well-tolerated intervention that has promise for slowing cognitive decline. The effectiveness of CCT is often affected by a lack of effective engagement. Mental fatigue is a the primary factor for compromising effective engagement in CCT, particularly in older adults at risk for dementia. There is a need for scalable, automated measures that can constantly monitor and reliably detect mental fatigue during CCT. Here, we develop and validate a novel Recurrent Video Transformer (RVT) method for monitoring real-time mental fatigue in older adults with mild cognitive impairment using their video-recorded facial gestures during CCT. The RVT model achieved the highest balanced accuracy (79.58%) and precision (0.82) compared to the prior models for binary and multi-class classification of mental fatigue. We also validated our model by significantly relating to reaction time across CCT tasks (Waldχ2=5.16,p=0.023). By leveraging dynamic temporal information, the RVT model demonstrates the potential to accurately measure real-time mental fatigue, laying the foundation for future CCT research aiming to enhance effective engagement by timely prevention of mental fatigue.

The Five Basic Human Senses Evoke Electrodermal Activity

Electrodermal activity (EDA) usually relates to variations in the electrical properties of palmar or plantar skin sites. EDA responses, namely skin conductance responses (SCRs), skin potential responses (SPRs) and skin susceptance responses (SSRs) are shown to be sensitive indexes of sympathetic nervous system activation and are studied in many research projects. However, the association between EDA responses and the five basic human senses has not been investigated yet. Our study aimed to explore the relationship between the three EDA responses (SCRs, SSRs and SPRs) and the five basic human senses. These three EDA responses were measured simultaneously at the same skin site on each of the 38 volunteers. The tested five senses were sight, hearing, touch, taste and smell. The results showed that the different tested senses led to different degrees of EDA responses due to activation of the sympathetic nervous system and corresponding secretion of sweat. Although a controlled study on the degree of EDA as a function of the strength of each stimulus was not performed, we noted that the largest EDA responses were typically associated with the smell sense test. We conclude that EDA responses could be utilized as measures for examining the sensitivity of the human senses. Hence, EDA devices may have important roles in sensory systems for future clinical applications.