On Assessing Driver Awareness of Situational Criticalities: Multi-modal Bio-Sensing and Vision-Based Analysis, Evaluations, and Insights

Remote Heart Rate Prediction in Virtual Reality Head-Mounted Displays Using Machine Learning Techniques

Head-mounted displays are virtual reality devices that may be equipped with sensors and cameras to measure a patient's heart rate through facial regions. Heart rate is an essential body signal that can be used to remotely monitor users in a variety of situations. There is currently no study that predicts heart rate using only highlighted facial regions; thus, an adaptation is required for beats per minute predictions. Likewise, there are no datasets containing only the eye and lower face regions, necessitating the development of a simulation mechanism. This work aims to remotely estimate heart rate from facial regions that can be captured by the cameras of a head-mounted display using state-of-the-art EVM-CNN and Meta-rPPG techniques. We developed a region of interest extractor to simulate a dataset from a head-mounted display device using stabilizer and video magnification techniques. Then, we combined support vector machine and FaceMash to determine the regions of interest and adapted photoplethysmography and beats per minute signal predictions to work with the other techniques. We observed an improvement of 188.88% for the EVM and 55.93% for the Meta-rPPG. In addition, both models were able to predict heart rate using only facial regions as input. Moreover, the adapted technique Meta-rPPG outperformed the original work, whereas the EVM adaptation produced comparable results for the photoplethysmography signal.

Machine Learning Techniques for Arousal Classification from Electrodermal Activity: A Systematic Review

This article introduces a systematic review on arousal classification based on electrodermal activity (EDA) and machine learning (ML). From a first set of 284 articles searched for in six scientific databases, fifty-nine were finally selected according to various criteria established. The systematic review has made it possible to analyse all the steps to which the EDA signals are subjected: acquisition, pre-processing, processing and feature extraction. Finally, all ML techniques applied to the features of these signals for arousal classification have been studied. It has been found that support vector machines and artificial neural networks stand out within the supervised learning methods given their high-performance values. In contrast, it has been shown that unsupervised learning is not present in the detection of arousal through EDA. This systematic review concludes that the use of EDA for the detection of arousal is widely spread, with particularly good results in classification with the ML methods found.

Machine Learning Models and Videos of Facial Regions for Estimating Heart Rate: A Review on Patents, Datasets, and Literature

Estimating heart rate is important for monitoring users in various situations. Estimates based on facial videos are increasingly being researched because they allow the monitoring of cardiac information in a non-invasive way and because the devices are simpler, as they require only cameras that capture the user’s face. From these videos of the user’s face, machine learning can estimate heart rate. This study investigates the benefits and challenges of using machine learning models to estimate heart rate from facial videos through patents, datasets, and article review. We have searched the Derwent Innovation, IEEE Xplore, Scopus, and Web of Science knowledge bases and identified seven patent filings, eleven datasets, and twenty articles on heart rate, photoplethysmography, or electrocardiogram data. In terms of patents, we note the advantages of inventions related to heart rate estimation, as described by the authors. In terms of datasets, we have discovered that most of them are for academic purposes and with different signs and annotations that allow coverage for subjects other than heartbeat estimation. In terms of articles, we have discovered techniques, such as extracting regions of interest for heart rate reading and using video magnification for small motion extraction, and models, such as EVM-CNN and VGG-16, that extract the observed individual’s heart rate, the best regions of interest for signal extraction, and ways to process them.