Estimation of Mental Distress from Photoplethysmography

A Deep Learning-Based Platform for Workers' Stress Detection Using Minimally Intrusive Multisensory Devices

The advent of Industry 4.0 necessitates substantial interaction between humans and machines, presenting new challenges when it comes to evaluating the stress levels of workers who operate in increasingly intricate work environments. Undoubtedly, work-related stress exerts a significant influence on individuals' overall stress levels, leading to enduring health issues and adverse impacts on their quality of life. Although psychological questionnaires have traditionally been employed to assess stress, they lack the capability to monitor stress levels in real-time or on an ongoing basis, thus making it arduous to identify the causes and demanding aspects of work. To surmount this limitation, an effective solution lies in the analysis of physiological signals that can be continuously measured through wearable or ambient sensors. Previous studies in this field have mainly focused on stress assessment through intrusive wearable systems susceptible to noise and artifacts that degrade performance. One of our recently published papers presented a wearable and ambient hardware-software platform that is minimally intrusive, able to detect human stress without hindering normal work activities, and slightly susceptible to artifacts due to movements. A limitation of this system is its not very high performance in terms of the accuracy of detecting multiple stress levels; therefore, in this work, the focus was on improving the software performance of the platform, using a deep learning approach. To this purpose, three neural networks were implemented, and the best performance was achieved by the 1D-convolutional neural network with an accuracy of 95.38% for the identification of two levels of stress, which is a significant improvement over those obtained previously.

The 2023 wearable photoplethysmography roadmap

Photoplethysmography is a key sensing technology which is used in wearable devices such as smartwatches and fitness trackers. Currently, photoplethysmography sensors are used to monitor physiological parameters including heart rate and heart rhythm, and to track activities like sleep and exercise. Yet, wearable photoplethysmography has potential to provide much more information on health and wellbeing, which could inform clinical decision making. This Roadmap outlines directions for research and development to realise the full potential of wearable photoplethysmography. Experts discuss key topics within the areas of sensor design, signal processing, clinical applications, and research directions. Their perspectives provide valuable guidance to researchers developing wearable photoplethysmography technology.

Mental Stress Detection Using a Wearable In-Ear Plethysmography

This study presents an ear-mounted photoplethysmography (PPG) system that is designed to detect mental stress. Mental stress is a prevalent condition that can negatively impact an individual's health and well-being. Early detection and treatment of mental stress are crucial for preventing related illnesses and maintaining overall wellness. The study used data from 14 participants that were collected in a controlled environment. The participants were subjected to stress-inducing tasks such as the Stroop color-word test and mathematical calculations. The raw PPG signal was then preprocessed and transformed into scalograms using continuous wavelet transform (CWT). A convolutional neural network classifier was then used to classify the transformed signals as stressed or non-stressed. The results of the study show that the PPG system achieved high levels of accuracy (92.04%) and F1-score (90.8%). Furthermore, by adding white Gaussian noise to the raw PPG signals, the results were improved even more, with an accuracy of 96.02% and an F1-score of 95.24%. The proposed ear-mounted device shows great promise as a reliable tool for the early detection and treatment of mental stress, potentially revolutionizing the field of mental health and well-being.

AI-Enabled Smart Wristband Providing Real-Time Vital Signs and Stress Monitoring

This work introduces the design, architecture, implementation, and testing of a low-cost and machine-learning-enabled device to be worn on the wrist. The suggested wearable device has been developed for use during emergency incidents of large passenger ship evacuations, and enables the real-time monitoring of the passengers' physiological state, and stress detection. Based on a properly preprocessed PPG signal, the device provides essential biometric data (pulse rate and oxygen saturation level) and an efficient unimodal machine learning pipeline. The stress detecting machine learning pipeline is based on ultra-short-term pulse rate variability, and has been successfully integrated into the microcontroller of the developed embedded device. As a result, the presented smart wristband is able to provide real-time stress detection. The stress detection system has been trained with the use of the publicly available WESAD dataset, and its performance has been tested through a two-stage process. Initially, evaluation of the lightweight machine learning pipeline on a previously unseen subset of the WESAD dataset was performed, reaching an accuracy score equal to 91%. Subsequently, external validation was conducted, through a dedicated laboratory study of 15 volunteers subjected to well-acknowledged cognitive stressors while wearing the smart wristband, which yielded an accuracy score equal to 76%.

Photoplethysmography Enabled Wearable Devices and Stress Detection: A Scoping Review

BACKGROUND

Mental and physical health are both important for overall health. Mental health includes emotional, psychological, and social well-being; however, it is often difficult to monitor remotely. The objective of this scoping review is to investigate studies that focus on mental health and stress detection and monitoring using PPG-based wearable sensors.

METHODS

A literature review for this scoping review was conducted using the PRISMA (Preferred Reporting Items for the Systematic Reviews and Meta-analyses) framework. A total of 290 studies were found in five medical databases (PubMed, Medline, Embase, CINAHL, and Web of Science). Studies were deemed eligible if non-invasive PPG-based wearables were worn on the wrist or ear to measure vital signs of the heart (heart rate, pulse transit time, pulse waves, blood pressure, and blood volume pressure) and analyzed the data qualitatively.

RESULTS

Twenty-three studies met the inclusion criteria, with four real-life studies, eighteen clinical studies, and one joint clinical and real-life study. Out of the twenty-three studies, seventeen were published as journal-based articles, and six were conference papers with full texts. Because most of the articles were concerned with physiological and psychological stress, we decided to only include those that focused on stress. In twelve of the twenty articles, a PPG-based sensor alone was used to monitor stress, while in the remaining eight papers, a PPG sensor was used in combination with other sensors.

CONCLUSION

The growing demand for wearable devices for mental health monitoring is evident. However, there is still a significant amount of research required before wearable devices can be used easily and effectively for such monitoring. Although the results of this review indicate that mental health monitoring and stress detection using PPG is possible, there are still many limitations within the current literature, such as a lack of large and diverse studies and ground-truth methods, that need to be addressed before wearable devices can be globally useful to patients.

Wearable Photoplethysmography for Cardiovascular Monitoring

Smart wearables provide an opportunity to monitor health in daily life and are emerging as potential tools for detecting cardiovascular disease (CVD). Wearables such as fitness bands and smartwatches routinely monitor the photoplethysmogram signal, an optical measure of the arterial pulse wave that is strongly influenced by the heart and blood vessels. In this survey, we summarize the fundamentals of wearable photoplethysmography and its analysis, identify its potential clinical applications, and outline pressing directions for future research in order to realize its full potential for tackling CVD.

Physical Exercise Effects on University Students’ Attention: An EEG Analysis Approach

Physically active breaks (AB) are currently being proposed as an interesting tool to improve students’ attention. Reviews and meta-analyses confirm their effect on attention, but also warned about the sparse evidence based on vigilance and university students. Therefore, this pilot study aimed to (a) determine the effects of AB in comparison with passive breaks on university students’ vigilance and (b) to validate an analysis model based on machine learning algorithms in conjunction with a multiparametric model based on electroencephalography (EEG) signal features. Through a counterbalanced within-subject experimental study, six university students (two female; mean age = 25.67, STD = 3.61) had their vigilance performances (i.e., response time in Psycho-Motor Vigilance Task) and EEG measured, before and after a lecture with an AB and another lecture with a passive break. A multiparametric model based on the spectral power, signal entropy and response time has been developed. Furthermore, this model, together with different machine learning algorithms, shows that for the taken signals there are significant differences after the AB lesson, implying an improvement in attention. These differences are most noticeable with the SVM with RBF kernel and ANNs with F1-score of 85% and 88%, respectively. In conclusion, results showed that students performed better on vigilance after the lecture with AB. Although limited, the evidence found could help researchers to be more accurate in their EEG analyses and lecturers and teachers to improve their students’ attentions in a proper way.

Photoplethysmogram Analysis and Applications: An Integrative Review

Beyond its use in a clinical environment, photoplethysmogram (PPG) is increasingly used for measuring the physiological state of an individual in daily life. This review aims to examine existing research on photoplethysmogram concerning its generation mechanisms, measurement principles, clinical applications, noise definition, pre-processing techniques, feature detection techniques, and post-processing techniques for photoplethysmogram processing, especially from an engineering point of view. We performed an extensive search with the PubMed, Google Scholar, Institute of Electrical and Electronics Engineers (IEEE), ScienceDirect, and Web of Science databases. Exclusion conditions did not include the year of publication, but articles not published in English were excluded. Based on 118 articles, we identified four main topics of enabling PPG: (A) PPG waveform, (B) PPG features and clinical applications including basic features based on the original PPG waveform, combined features of PPG, and derivative features of PPG, (C) PPG noise including motion artifact baseline wandering and hypoperfusion, and (D) PPG signal processing including PPG preprocessing, PPG peak detection, and signal quality index. The application field of photoplethysmogram has been extending from the clinical to the mobile environment. Although there is no standardized pre-processing pipeline for PPG signal processing, as PPG data are acquired and accumulated in various ways, the recently proposed machine learning-based method is expected to offer a promising solution.

Efficient methods for acute stress detection using heart rate variability data from Ambient Assisted Living sensors

BACKGROUND

Using Ambient Assisted Living sensors to detect acute stress could help people mitigate the harmful effects of everyday stressful situations. This would help both the healthy and those affected more by sudden stressors, e.g., people with diabetes or heart conditions. The study aimed to develop a method for providing reliable stress detection based on heart rate variability features extracted from portable devices.

METHODS

Features extracted from portable electrocardiogram sensor recordings were used for training various classification algorithms for stress detection purposes. Data were recorded in a clinical trial with 7 participants and two stressors, the Trier Social Stress Test and the Stroop colour word test, both validated by standardised questionnaires. Different heart rate variability feature sets (all, time-domain and non-linear only, frequency-domain only) were tested to investigate how classification performance is affected, in addition to various time window length setups and participant-wise training sessions. The accuracy and F1 score of the trained models were compared and analysed.

RESULTS

The best results were achieved with models using time-domain and non-linear heart rate variability features with 5-min-long overlapping time windows, yielding 96.31% accuracy and 96.26% F1 score. Shorter overlapping windows had slightly lower performance, with 91.62-94.55% accuracy and 91.77-94.55% F1 score ranges. Non-overlapping window configurations were less effective, with both accuracy and F1 score below 88%. For participant-wise learning, average F1 scores of 99.47%, 98.93% and 96.1% were achieved for feature sets using all, time-domain and non-linear, and frequency-domain features, respectively.

CONCLUSION

The tested stress detector models based on heart rate variability data recorded by a single electrocardiogram sensor performed just as well as those published in the literature working with multiple sensors, or even better. This suggests that once portable devices such as smartwatches provide reliable hear rate variability recordings, efficient stress detection can be achieved without the need for additional physiological measurements.

Stress Classification Using Photoplethysmogram-Based Spatial and Frequency Domain Images

Stress is subjective and is manifested differently from one person to another. Thus, the performance of generic classification models that classify stress status is crude. Building a person-specific model leads to a reliable classification, but it requires the collection of new data to train a new model for every individual and needs periodic upgrades because stress is dynamic. In this paper, a new binary classification (called stressed and non-stressed) approach is proposed for a subject’s stress state in which the inter-beat intervals extracted from a photoplethysomogram (PPG) were transferred to spatial images and then to frequency domain images according to the number of consecutive. Then, the convolution neural network (CNN) was used to train and validate the classification accuracy of the person’s stress state. Three types of classification models were built: person-specific models, generic classification models, and calibrated-generic classification models. The average classification accuracies achieved by person-specific models using spatial images and frequency domain images were 99.9%, 100%, and 99.8%, and 99.68%, 98.97%, and 96.4% for the training, validation, and test, respectively. By combining 20% of the samples collected from test subjects into the training data, the calibrated generic models’ accuracy was improved and outperformed the generic performance across both the spatial and frequency domain images. The average classification accuracy of 99.6%, 99.9%, and 88.1%, and 99.2%, 97.4%, and 87.6% were obtained for the training set, validation set, and test set, respectively, using the calibrated generic classification-based method for the series of inter-beat interval (IBI) spatial and frequency domain images. The main contribution of this study is the use of the frequency domain images that are generated from the spatial domain images of the IBI extracted from the PPG signal to classify the stress state of the individual by building person-specific models and calibrated generic models.

Differences between young and older adults in physiological and subjective responses to emotion induction using films

Emotional response in aging is typically studied using the dimensional or the discrete models of emotion. Moreover, it is typically studied using subjective or physiological variables but not using both perspectives simultaneously. Additionally, tenderness is neglected in emotion induction procedures with older adults, with the present work being the first to include the study of physiological tenderness using film clips. This study integrated two separate approaches to emotion research, comparing 68 younger and 39 older adults and using a popular set of film clips to induce tenderness, amusement, anger, fear, sadness and disgust emotions. The direction of subjective emotional patterns was evaluated with self-reports and that of physiological emotional patterns was evaluated with a wearable emotion detection system. The findings suggest a dual-process framework between subjective and physiological responses, manifested differently in young and older adults. In terms of arousal, the older adults exhibited higher levels of subjective arousal in negative emotions and tenderness while young adults showed higher levels of physiological arousal in these emotions. These findings yield information on the multidirectionality of positive and negative emotions, corroborating that emotional changes in the adult lifespan appear to be subject to the relevance of the emotion elicitor to each age group.

Influence of mental stress on the pulse wave features of photoplethysmograms

Mental stress is a major burden for our society. Invasive and non-invasive methods have been proposed to monitor and quantify it using various sensors on and off body. In this Letter, the authors investigated the use of the arm photoplethysmogram (PPG) to assess mental stress in laboratory conditions. Results were in correspondence with their previous in-silico study which guided the present study. Three wave shape parameters were identified for stress assessment from the PPG signal: (i) the time from dicrotic notch to end diastole; (ii) the time from pulse onset to systolic peak; and (iii) the ratio of diastolic to systolic area. The proposed in-vivo results showed that the two first parameters responded significantly to increased mental stress and to a breathing relaxation procedure, complementing heart rate, heart rate variability, and pulse transit time as indices of stress.

Remote heart rate monitoring - Assessment of the Facereader rPPg by Noldus

Remote photoplethysmography (rPPG) allows contactless monitoring of human cardiac activity through a video camera. In this study, we assessed the accuracy and precision for heart rate measurements of the only consumer product available on the market, namely the FacereaderTM rPPG by Noldus, with respect to a gold standard electrocardiograph. Twenty-four healthy participants were asked to sit in front of a computer screen and alternate two periods of rest with two stress tests (i.e. Go/No-Go task), while their heart rate was simultaneously acquired for 20 minutes using the ECG criterion measure and the FacereaderTM rPPG. Results show that the FacereaderTM rPPG tends to overestimate lower heart rates and underestimate higher heart rates compared to the ECG. The Facereader™ rPPG revealed a mean bias of 9.8 bpm, the 95% limits of agreement (LoA) ranged from almost -30 up to +50 bpm. These results suggest that whilst the rPPG FacereaderTM technology has potential for contactless heart rate monitoring, its predictions are inaccurate for higher heart rates, with unacceptable precision across the entire range, rendering its estimates unreliable for monitoring individuals.

An Empirical Study Comparing Unobtrusive Physiological Sensors for Stress Detection in Computer Work

Several unobtrusive sensors have been tested in studies to capture physiological reactions to stress in workplace settings. Lab studies tend to focus on assessing sensors during a specific computer task, while in situ studies tend to offer a generalized view of sensors' efficacy for workplace stress monitoring, without discriminating different tasks. Given the variation in workplace computer activities, this study investigates the efficacy of unobtrusive sensors for stress measurement across a variety of tasks. We present a comparison of five physiological measurements obtained in a lab experiment, where participants completed six different computer tasks, while we measured their stress levels using a chest-band (ECG, respiration), a wristband (PPG and EDA), and an emerging thermal imaging method (perinasal perspiration). We found that thermal imaging can detect increased stress for most participants across all tasks, while wrist and chest sensors were less generalizable across tasks and participants. We summarize the costs and benefits of each sensor stream, and show how some computer use scenarios present usability and reliability challenges for stress monitoring with certain physiological sensors. We provide recommendations for researchers and system builders for measuring stress with physiological sensors during workplace computer use.

Body Area Networks in Healthcare: A Brief State of the Art

A body area network (BAN) comprises a set of devices that sense their surroundings, activate and communicate with each other when an event is detected in its environment. Although BAN technology was developed more than 20 years ago, in recent years, its popularity has greatly increased. The reason is the availability of smaller and more powerful devices, more efficient communication protocols and improved duration of portable batteries. BANs are applied in many fields, healthcare being one of the most important through gathering information about patients and their surroundings. A continuous stream of information may help physicians with making well-informed decisions about a patient’s treatment. Based on recent literature, the authors review BAN architectures, network topologies, energy sources, sensor types, applications, as well as their main challenges. In addition, the paper focuses on the principal requirements of safety, security, and sustainability. In addition, future research and improvements are discussed.

Assessing mental stress from the photoplethysmogram: a numerical study

OBJECTIVE

Mental stress is detrimental to cardiovascular health, being a risk factor for coronary heart disease and a trigger for cardiac events. However, it is not currently routinely assessed. The aim of this study was to identify features of the photoplethysmogram (PPG) pulse wave which are indicative of mental stress.

APPROACH

A numerical model of pulse wave propagation was used to simulate blood pressure signals, from which simulated PPG pulse waves were estimated using a transfer function. Pulse waves were simulated at six levels of stress by changing the model input parameters both simultaneously and individually, in accordance with haemodynamic changes associated with stress. Thirty-two feature measurements were extracted from pulse waves at three measurement sites: the brachial, radial and temporal arteries. Features which changed significantly with stress were identified using the Mann-Kendall monotonic trend test.

MAIN RESULTS

Seventeen features exhibited significant trends with stress in measurements from at least one site. Three features showed significant trends at all three sites: the time from pulse onset to peak, the time from the dicrotic notch to pulse end, and the pulse rate. More features showed significant trends at the radial artery (15) than the brachial (8) or temporal (7) arteries. Most features were influenced by multiple input parameters.

SIGNIFICANCE

The features identified in this study could be used to monitor stress in healthcare and consumer devices. Measurements at the radial artery may provide superior performance than the brachial or temporal arteries. In vivo studies are required to confirm these observations.