A Review of Deep Learning-Based Contactless Heart Rate Measurement Methods

High-accuracy heart rate detection using multispectral IPPG technology combined with a deep learning algorithm

Image Photoplethysmography (IPPG) technology is a noncontact physiological parameter detection technology, which has been widely used in heart rate (HR) detection. However, traditional imaging devices still have issues such as narrower receiving spectral range and inferior motion detection performance. In this paper, we propose a HR detection method based on multi-spectral video. Our method combining multispectral imaging with IPPG technology provides more accurate physiological information. To realize real-time evaluation of HR directly from facial multispectral videos, we propose a new end-to-end neural network, namely IPPGResNet18. The IPPGResNet18 model was trained on the multispectral video dataset from which better results were achieved: MAE = 2.793, RMSE = 3.695, SD = 3.707, p = 0.304. The experimental results demonstrate a high accuracy of HR detection under motion state using this detection method. In respect of real-time monitoring of HR during movement, our method is obviously superior to the conventional technical solutions.

Evaluation of transformation invariant loss function with distance equilibrium in prediction of imaging photoplethysmography characteristics

Objective. Monitoring changes in human heart rate variability (HRV) holds significant importance for protecting life and health. Studies have shown that Imaging Photoplethysmography (IPPG) based on ordinary color cameras can detect the color change of the skin pixel caused by cardiopulmonary system. Most researchers employed deep learning IPPG algorithms to extract the blood volume pulse (BVP) signal, analyzing it predominantly through the heart rate (HR). However, this approach often overlooks the inherent intricate time-frequency domain characteristics in the BVP signal, which cannot be comprehensively deduced solely from HR. The analysis of HRV metrics through the BVP signal is imperative.

APPROACH

In this paper, the transformation invariant loss function with distance equilibrium (TIDLE) loss function is applied to IPPG for the first time, and the details of BVP signal can be recovered better. In detail, TIDLE is tested in four commonly used IPPG deep learning models, which are DeepPhys, EfficientPhys, Physnet and TS_CAN, and compared with other three loss functions, which are mean absolute error (MAE), mean square error (MSE), Neg Pearson Coefficient correlation (NPCC).

MAIN RESULTS

The experiments demonstrate that MAE and MSE exhibit suboptimal performance in predicting LF/HF across the four models, achieving the Statistic of Mean Absolute Error (MAES) of 25.94% and 34.05%, respectively. In contrast, NPCC and TIDLE yielded more favorable results at 13.51% and 11.35%, respectively. Taking into consideration the morphological characteristics of the BVP signal, on the two optimal models for predicting HRV metrics, namely DeepPhys and TS_CAN, the Pearson coefficients for the BVP signals predicted by TIDLE in comparison to the gold-standard BVP signals achieved values of 0.627 and 0.605, respectively. In contrast, the results based on NPCC were notably lower, at only 0.545 and 0.533, respectively.

SIGNIFICANCE

This paper contributes significantly to the effective restoration of the morphology and frequency domain characteristics of the BVP signal.

Conventional and deep learning methods in heart rate estimation from RGB face videos

Contactless vital signs monitoring is a fast-advancing scientific field that aims to employ monitoring methods that do not necessitate the use of leads or physical attachments to the patient in order to overcome the shortcomings and limits of traditional monitoring systems. Several traditional methods have been applied to extract the heart rate (HR) signal from the face. Moreover, machine learning has recently contributed majorly to the development of such a field in which deep networks and other deep learning methods are employed to extract the HR signal from RGB face videos. In this paper, we evaluate the state-of-the-art conventional and deep learning methods for HR estimates, focusing on the limits of deep learning methods and the availability of less-controlled face video datasets. We aim to present an extensive review that helps the various approaches of remote photoplethysmography extraction and HR estimation to be understood, in addition to their drawbacks and benefits.

A System for Monitoring Animals Based on Behavioral Information and Internal State Information

Managing the risk of injury or illness is an important consideration when keeping pets. This risk can be minimized if pets are monitored on a regular basis, but this can be difficult and time-consuming. However, because only the external behavior of the animal can be observed and the internal condition cannot be assessed, the animal's state can easily be misjudged. Additionally, although some systems use heartbeat measurement to determine a state of tension, or use rest to assess the internal state, because an increase in heart rate can also occur as a result of exercise, it is desirable to use this measurement in combination with behavioral information. In the current study, we proposed a monitoring system for animals using video image analysis. The proposed system first extracts features related to behavioral information and the animal's internal state via mask R-CNN using video images taken from the top of the cage. These features are used to detect typical daily activities and anomalous activities. This method produces an alert when the hamster behaves in an unusual way. In our experiment, the daily behavior of a hamster was measured and analyzed using the proposed system. The results showed that the features of the hamster's behavior were successfully detected. When loud sounds were presented from outside the cage, the system was able to discriminate between the behavioral and internal changes of the hamster. In future research, we plan to improve the accuracy of the measurement of small movements and develop a more accurate system.

Capturing the pulse: a state-of-the-art review on camera-based jugular vein assessment

Heart failure is associated with a rehospitalisation rate of up to 50% within six months. Elevated central venous pressure may serve as an early warning sign. While invasive procedures are used to measure central venous pressure for guiding treatment in hospital, this becomes impractical upon discharge. A non-invasive estimation technique exists, where the clinician visually inspects the pulsation of the jugular veins in the neck, but it is less reliable due to human limitations. Video and signal processing technologies may offer a high-fidelity alternative. This state-of-the-art review analyses existing literature on camera-based methods for jugular vein assessment. We summarize key design considerations and suggest avenues for future research. Our review highlights the neck as a rich imaging target beyond the jugular veins, capturing comprehensive cardiac signals, and outlines factors affecting signal quality and measurement accuracy. Addressing an often quoted limitation in the field, we also propose minimum reporting standards for future studies.

Face2PPG: An Unsupervised Pipeline for Blood Volume Pulse Extraction From Faces

Photoplethysmography (PPG) signals have become a key technology in many fields, such as medicine, well-being, or sports. Our work proposes a set of pipelines to extract remote PPG signals (rPPG) from the face robustly, reliably, and configurably. We identify and evaluate the possible choices in the critical steps of unsupervised rPPG methodologies. We assess a state-of-the-art processing pipeline in six different datasets, incorporating important corrections in the methodology that ensure reproducible and fair comparisons. In addition, we extend the pipeline by proposing three novel ideas; 1) a new method to stabilize the detected face based on a rigid mesh normalization; 2) a new method to dynamically select the different regions in the face that provide the best raw signals, and 3) a new RGB to rPPG transformation method, called Orthogonal Matrix Image Transformation (OMIT) based on QR decomposition, that increases robustness against compression artifacts. We show that all three changes introduce noticeable improvements in retrieving rPPG signals from faces, obtaining state-of-the-art results compared with unsupervised, non-learning-based methodologies and, in some databases, very close to supervised, learning-based methods. We perform a comparative study to quantify the contribution of each proposed idea. In addition, we depict a series of observations that could help in future implementations.

Frugal Heart Rate Correction Method for Scalable Health and Safety Monitoring in Construction Sites

Continuous, real-time monitoring of occupational health and safety in high-risk workplaces such as construction sites can substantially improve the safety of workers. However, introducing such systems in practice is associated with a number of challenges, such as scaling up the solution while keeping its cost low. In this context, this work investigates the use of an off-the-shelf, low-cost smartwatch to detect health issues based on heart rate monitoring in a privacy-preserving manner. To improve the smartwatch's low measurement quality, a novel, frugal machine learning method is proposed that corrects measurement errors, along with a new dataset for this task. This method's integration with the smartwatch and the remaining parts of the health and safety monitoring system (built on the ASSIST-IoT reference architecture) are presented. This method was evaluated in a laboratory environment in terms of its accuracy, computational requirements, and frugality. With an experimentally established mean absolute error of 8.19 BPM, only 880 bytes of required memory, and a negligible impact on the performance of the device, this method meets all relevant requirements and is expected to be field-tested in the coming months. To support reproducibility and to encourage alternative approaches, the dataset, the trained model, and its implementation on the smartwatch were published under free licenses.

Photoplethysmography upon cold stress-impact of measurement site and acquisition mode

Photoplethysmography (PPG) allows various statements about the physiological state. It supports multiple recording setups, i.e., application to various body sites and different acquisition modes, rendering the technique a versatile tool for various situations. Owing to anatomical, physiological and metrological factors, PPG signals differ with the actual setup. Research on such differences can deepen the understanding of prevailing physiological mechanisms and path the way towards improved or novel methods for PPG analysis. The presented work systematically investigates the impact of the cold pressor test (CPT), i.e., a painful stimulus, on the morphology of PPG signals considering different recording setups. Our investigation compares contact PPG recorded at the finger, contact PPG recorded at the earlobe and imaging PPG (iPPG), i.e., non-contact PPG, recorded at the face. The study bases on own experimental data from 39 healthy volunteers. We derived for each recording setup four common morphological PPG features from three intervals around CPT. For the same intervals, we derived blood pressure and heart rate as reference. To assess differences between the intervals, we used repeated measures ANOVA together with paired t-tests for each feature and we calculated Hedges' g to quantify effect sizes. Our analyses show a distinct impact of CPT. As expected, blood pressure shows a highly significant and persistent increase. Independently of the recording setup, all PPG features show significant changes upon CPT as well. However, there are marked differences between recording setups. Effect sizes generally differ with the finger PPG showing the strongest response. Moreover, one feature (pulse width at half amplitude) shows an inverse behavior in finger PPG and head PPG (earlobe PPG and iPPG). In addition, iPPG features behave partially different from contact PPG features as they tend to return to baseline values while contact PPG features remain altered. Our findings underline the importance of recording setup and physiological as well as metrological differences that relate to the setups. The actual setup must be considered in order to properly interpret features and use PPG. The existence of differences between recording setups and a deepened knowledge on such differences might open up novel diagnostic methods in the future.

Contactless Camera-Based Heart Rate and Respiratory Rate Monitoring Using AI on Hardware

Detecting vital signs by using a contactless camera-based approach can provide several advantages over traditional clinical methods, such as lower financial costs, reduced visit times, increased comfort, and enhanced safety for healthcare professionals. Specifically, Eulerian Video Magnification (EVM) or Remote Photoplethysmography (rPPG) methods can be utilised to remotely estimate heart rate and respiratory rate biomarkers. In this paper two contactless camera-based health monitoring architectures are developed using EVM and rPPG, respectively; to this end, two different CNNs, (Mediapipe's BlazeFace and FaceMesh) are used to extract suitable regions of interest from incoming video frames. These two methods are implemented and deployed on four off-the-shelf edge devices as well as on a PC and evaluated in terms of latency (in each stage of the application's pipeline), throughput (FPS), power consumption (Watt), efficiency (throughput/Watt), and value (throughput/cost). This work provides important insights about the computational costs and bottlenecks of each method on each hardware platform, as well as which platform to use depending on the target metric. One of our insights shows that the Jetson Xavier NX platform is the best platform in terms of throughput and efficiency, while Raspberry Pi 4 8 GB is the best platform in terms of value.

An Evaluation of Non-Contact Photoplethysmography-Based Methods for Remote Respiratory Rate Estimation

The respiration rate (RR) is one of the physiological signals deserving monitoring for assessing human health and emotional states. However, traditional devices, such as the respiration belt to be worn around the chest, are not always a feasible solution (e.g., telemedicine, device discomfort). Recently, novel approaches have been proposed aiming at estimating RR in a less invasive yet reliable way, requiring the acquisition and processing of contact or remote Photoplethysmography (contact reference and remote-PPG, respectively). The aim of this paper is to address the lack of systematic evaluation of proposed methods on publicly available datasets, which currently impedes a fair comparison among them. In particular, we evaluate two prominent families of PPG processing methods estimating Respiratory Induced Variations (RIVs): the first encompasses methods based on the direct extraction of morphological features concerning the RR; and the second group includes methods modeling respiratory artifacts adopting, in the most promising cases, single-channel blind source separation. Extensive experiments have been carried out on the public BP4D+ dataset, showing that the morphological estimation of RIVs is more reliable than those produced by a single-channel blind source separation method (both in contact and remote testing phases), as well as in comparison with a representative state-of-the-art Deep Learning-based approach for remote respiratory information estimation.

X-iPPGNet: A novel one stage deep learning architecture based on depthwise separable convolutions for video-based pulse rate estimation

Pulse rate (PR) is one of the most important markers for assessing a person's health. With the increasing demand for long-term health monitoring, much attention is being paid to contactless PR estimation using imaging photoplethysmography (iPPG). This non-invasive technique is based on the analysis of subtle changes in skin color. Despite efforts to improve iPPG, the existing algorithms are vulnerable to less-constrained scenarios (i.e., head movements, facial expressions, and environmental conditions). In this article, we propose a novel end-to-end spatio-temporal network, namely X-iPPGNet, for instantaneous PR estimation directly from facial video recordings. Unlike most existing systems, our model learns the iPPG concept from scratch without incorporating any prior knowledge or going through the extraction of blood volume pulse signals. Inspired by the Xception network architecture, color channel decoupling is used to learn additional photoplethysmographic information and to effectively reduce the computational cost and memory requirements. Moreover, X-iPPGNet predicts the pulse rate from a short time window (2 s), which has advantages with high and sharply fluctuating pulse rates. The experimental results revealed high performance under all conditions including head motions, facial expressions, and skin tone. Our approach significantly outperforms all current state-of-the-art methods on three benchmark datasets: MMSE-HR (MAE = 4.10 ; RMSE = 5.32 ; r = 0.85), UBFC-rPPG (MAE = 4.99 ; RMSE = 6.26 ; r = 0.67), MAHNOB-HCI (MAE = 3.17 ; RMSE = 3.93 ; r = 0.88).

Using Computer Vision to Track Facial Color Changes and Predict Heart Rate

The current technological advances have pushed the quantification of exercise intensity to new era of physical exercise sciences. Monitoring physical exercise is essential in the process of planning, applying, and controlling loads for performance optimization and health. A lot of research studies applied various statistical approaches to estimate various physiological indices, to our knowledge, no studies found to investigate the relationship of facial color changes and increased exercise intensity. The aim of this study was to develop a non-contact method based on computer vision to determine the heart rate and, ultimately, the exercise intensity. The method was based on analyzing facial color changes during exercise by using RGB, HSV, YCbCr, Lab, and YUV color models. Nine university students participated in the study (mean age = 26.88 ± 6.01 years, mean weight = 72.56 ± 14.27 kg, mean height = 172.88 ± 12.04 cm, six males and three females, and all white Caucasian). The data analyses were carried out separately for each participant (personalized model) as well as all the participants at a time (universal model). The multiple auto regressions, and a multiple polynomial regression model were designed to predict maximum heart rate percentage (maxHR%) from each color models. The results were analyzed and evaluated using Root Mean Square Error (RMSE), F-values, and R-square. The multiple polynomial regression using all participants exhibits the best accuracy with RMSE of 6.75 (R-square = 0.78). Exercise prescription and monitoring can benefit from the use of these methods, for example, to optimize the process of online monitoring, without having the need to use any other instrumentation.

A deep learning approach to estimate pulse rate by remote photoplethysmography

OBJECTIVE

This study proposes an U-net shaped Deep Neural Network (DNN) model to extract remote photoplethysmography (rPPG) signals from skin color signals to estimate Pulse Rate (PR).

APPROACH

Three input window sizes are used into the DNN: 256 samples (5.12 s), 512 samples (10.24 s), and 1024 (20.48 s). A data argumentation algorithm based on interpolation is also used here to artificially increase the number of training samples.

MAIN RESULTS

The proposed model outperformed a prior-knowledge rPPG method by using input signals with window of 256 and 512 samples. Also, it was found that the data augmentation procedure only increased the performance for window of 1024 samples. The trained model achieved a Mean Absolute Error (MAE) of 3.97 Beats per Minute (BPM) and Root Mean Squared Error (RMSE) of 6.47 BPM, for the 256 samples window, and MAE of 3.00 BPM and RMSE of 5.45 BPM for the window of 512 samples. On the other hand, the prior-knowledge rPPG method got a MAE of 8.04 BPM and RMSE of 16.63 BPM for the window of 256 samples, and MAE of 3.49 BPM and RMSE of 7.92 BPM for the window of 512. For the longest window (1024 samples), the concordance of the predicted PRs from the DNNs and the true PRs was higher when applying the data augmentation procedure.

SIGNIFICANCE

These results demonstrate a big potential of this technique for PR estimation, showing that the DNN proposed here may generate reliable rPPG signals even with short window lengths (5.12 s and 10.24 s), suggesting that it needs less data for a faster rPPG measurement and PR estimation.

Heart rate prediction from facial video with masks using eye location and corrected by convolutional neural networks

Remote photoplethysmography (rPPG), which aims at measuring heart activities without any contact, has great potential in many applications. The emergence of novel coronavirus pneumonia COVID-19 has attracted worldwide attentions. Contact photoplethysmography (cPPG) methods need to contact the detection equipment with the patient, which may accelerate the spread of the epidemic. In the future, the non-contact heart rate detection will be an urgent need. However, existing heart rate measuring methods from facial videos are vulnerable to the less-constrained scenarios (e.g., with head movement and wearing a mask). In this paper, we proposed a method of heart rate detection based on eye location of region of interest (ROI) to solve the problem of missing information when wearing masks. Besides, a model to filter outliers based on residual network was conceived first by us and the better heart rate measurement accuracy was generated. To validate our method, we also created a mask dataset. The results demonstrated that after using our method for correcting the heart rate (HR) value measured with the traditional method, the accuracy reaches 4.65 bpm, which is 0.42 bpm higher than that without correction.

Stress Management Intervention for Leaders Increases Nighttime SDANN: Results from a Randomized Controlled Trial

Stress management interventions aim to reduce the disease risk that is heightened by work stress. Possible pathways of risk reduction include improvements in the autonomous nervous system, which is indexed by the measurement of heart rate variability (HRV). A randomized controlled trial on improving stress management skills at work was conducted to investigate the effects of intervention on HRV. A total of 174 lower management employees were randomized into either the waiting list control group (CG) or the intervention group (IG) receiving a 2-day stress management training program and another half-day booster after four and six months. In the trial, 24 h HRV was measured at baseline and after 12 months. Heart rate (HR), root mean square of successive differences (RMSSD), standard deviation of normal-to-normal intervals (SDNN), and standard deviation of the average of normal-to-normal intervals (SDANN) were calculated for 24 h and nighttime periods. Age-adjusted multilevel mixed effects linear regressions with unstructured covariance, time as a random coefficient, and time × group interaction with the according likelihood-ratio tests were calculated. The linear mixed-effect regression models showed neither group effects between IG and CG at baseline nor time effects between baseline and follow-up for SDANN (24 h), SDNN (24 h and nighttime), RMSSD (24 h and nighttime), and HR (24 h and nighttime). Nighttime SDANN significantly improved in the intervention group (z = 2.04, p = 0.041) compared to the control group. The objective stress axis measures (SDANN) showed successful stress reduction due to the training. Nighttime SDANN was strongly associated with minimum HR. Though the effects were small and only visible at night, it is highly remarkable that 3 days of intervention achieved a measurable effect considering that stress is only one of many factors that can influence HR and HRV.

An Embedded System Using Convolutional Neural Network Model for Online and Real-Time ECG Signal Classification and Prediction

This paper presents an automatic ECG signal classification system that applied the Deep Learning (DL) model to classify four types of ECG signals. In the first part of our work, we present the model development. Four different classes of ECG signals from the PhysioNet open-source database were selected and used. This preliminary study used a Deep Learning (DL) technique namely Convolutional Neural Network (CNN) to classify and predict the ECG signals from four different classes: normal, sudden death, arrhythmia, and supraventricular arrhythmia. The classification and prediction process includes pulse extraction, image reshaping, training dataset, and testing process. In general, the training accuracy achieved up to 95% after 100 epochs. However, the prediction of each ECG single type shows a differentiation. Among the four classes, the results show that the predictions for sudden death ECG waveforms are the highest, i.e., 80 out of 80 samples are correct (100% accuracy). In contrast, the lowest is the prediction for normal sinus ECG waveforms, i.e., 74 out of 80 samples are correct (92.5% accuracy). This is due to the image features of normal sinus ECG waveforms being almost similar to the image features of supraventricular arrhythmia ECG waveforms. However, the model has been tuned to achieve an optimal prediction. In the second part, we presented the hardware implementation with the predictive model embedded in an NVIDIA Jetson Nanoprocessor for the online and real-time classification of ECG waveforms.

Diagnostic Features and Potential Applications of PPG Signal in Healthcare: A Systematic Review

Recent research indicates that Photoplethysmography (PPG) signals carry more information than oxygen saturation level (SpO2) and can be utilized for affordable, fast, and noninvasive healthcare applications. All these encourage the researchers to estimate its feasibility as an alternative to many expansive, time-wasting, and invasive methods. This systematic review discusses the current literature on diagnostic features of PPG signal and their applications that might present a potential venue to be adapted into many health and fitness aspects of human life. The research methodology is based on the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines 2020. To this aim, papers from 1981 to date are reviewed and categorized in terms of the healthcare application domain. Along with consolidated research areas, recent topics that are growing in popularity are also discovered. We also highlight the potential impact of using PPG signals on an individual's quality of life and public health. The state-of-the-art studies suggest that in the years to come PPG wearables will become pervasive in many fields of medical practices, and the main domains include cardiology, respiratory, neurology, and fitness. Main operation challenges, including performance and robustness obstacles, are identified.

Contactless Vital Signs Monitoring From Videos Recorded With Digital Cameras: An Overview

The measurement of physiological parameters is fundamental to assess the health status of an individual. The contactless monitoring of vital signs may provide benefits in various fields of application, from healthcare and clinical setting to occupational and sports scenarios. Recent research has been focused on the potentiality of camera-based systems working in the visible range (380-750 nm) for estimating vital signs by capturing subtle color changes or motions caused by physiological activities but invisible to human eyes. These quantities are typically extracted from videos framing some exposed body areas (e.g., face, torso, and hands) with adequate post-processing algorithms. In this review, we provided an overview of the physiological and technical aspects behind the estimation of vital signs like respiratory rate, heart rate, blood oxygen saturation, and blood pressure from digital images as well as the potential fields of application of these technologies. Per each vital sign, we provided the rationale for the measurement, a classification of the different techniques implemented for post-processing the original videos, and the main results obtained during various applications or in validation studies. The available evidence supports the premise of digital cameras as an unobtrusive and easy-to-use technology for physiological signs monitoring. Further research is needed to promote the advancements of the technology, allowing its application in a wide range of population and everyday life, fostering a biometrical holistic of the human body (BHOHB) approach.

pyVHR: a Python framework for remote photoplethysmography

Remote photoplethysmography (rPPG) aspires to automatically estimate heart rate (HR) variability from videos in realistic environments. A number of effective methods relying on data-driven, model-based and statistical approaches have emerged in the past two decades. They exhibit increasing ability to estimate the blood volume pulse (BVP) signal upon which BPMs (Beats per Minute) can be estimated. Furthermore, learning-based rPPG methods have been recently proposed. The present pyVHR framework represents a multi-stage pipeline covering the whole process for extracting and analyzing HR fluctuations. It is designed for both theoretical studies and practical applications in contexts where wearable sensors are inconvenient to use. Namely, pyVHR supports either the development, assessment and statistical analysis of novel rPPG methods, either traditional or learning-based, or simply the sound comparison of well-established methods on multiple datasets. It is built up on accelerated Python libraries for video and signal processing as well as equipped with parallel/accelerated ad-hoc procedures paving the way to online processing on a GPU. The whole accelerated process can be safely run in real-time for 30 fps HD videos with an average speedup of around 5. This paper is shaped in the form of a gentle tutorial presentation of the framework.

A novel contact-free atrial fibrillation monitor: a pilot study

Aims

Atrial fibrillation (AF) is a major cause of morbidity and mortality. Current guidelines support performing electrocardiogram (ECG) screenings to spot AF in high-risk patients. The purpose of this study was to validate a new algorithm aimed to identify AF in patients measured with a recent FDA-cleared contact-free optical device.

Methods and results

Study participants were measured simultaneously using two devices: a contact-free optical system that measures chest motion vibrations (investigational device, 'Gili') and a standard reference bed-side ECG monitor (Mindray^®). Each reference ECG was evaluated by two board certified cardiologists that defined each trace as: regular rhythm, AF, other irregular rhythm or indecipherable/missing. A total of 3582, 30-s intervals, pertaining to 444 patients (41.9% with a history of AF) were made available for analysis. Distribution of patients with active AF, other irregular rhythm, and regular rhythm was 16.9%, 29.5%, and 53.6% respectively. Following application of cross-validated machine learning approach, the observed sensitivity and specificity were 0.92 [95% confidence interval (CI): 0.91-0.93] and 0.96 (95% CI: 0.95-0.96), respectively.

Conclusion

This study demonstrates for the first time the efficacy of a contact-free optical device for detecting AF.

Contactless Vital Sign Monitoring System for Heart and Respiratory Rate Measurements with Motion Compensation Using a Near-Infrared Time-of-Flight Camera

This study describes a contactless vital sign monitoring (CVSM) system capable of measuring heart rate (HR) and respiration rate (RR) using a low-power, indirect time-of-flight (ToF) camera. The system takes advantage of both the active infrared illumination as well as the additional depth information from the ToF camera to compensate for the motion-induced artifacts during the HR measurements. The depth information captures how the user is moving with respect to the camera and, therefore, can be used to differentiate where the intensity change in the raw signal is from the underlying heartbeat or motion. Moreover, from the depth information, the system can acquire respiration rate by directly measuring the motion of the chest wall during breathing. We also conducted a pilot human study using this system with 29 participants of different demographics such as age, gender, and skin color. Our study shows that with depth-based motion compensation, the success rate (system measurement within 10% of reference) of HR measurements increases to 75%, as compared to 35% when motion compensation is not used. The mean HR deviation from the reference also drops from 21 BPM to −6.25 BPM when we apply the depth-based motion compensation. In terms of the RR measurement, our system shows a mean deviation of 1.7 BPM from the reference measurement. The pilot human study shows the system performance is independent of skin color but weakly dependent on gender and age.

iPPG 2 cPPG: Reconstructing contact from imaging photoplethysmographic signals using U-Net architectures

Imaging photoplethysmography (iPPG) is an optical technique dedicated to the assessment of several vital functions using a simple camera. Significant efforts have been made to reliably estimate heart and respiratory rates. Currently, research is focusing on the remote estimation of oxygen saturation and blood pressure (BP). The limited number of publicly available data tends to restrict the advancements related to BP estimation. To overcome this limit, we propose to split the problem in a two-stage processing chain: (i) converting iPPG to contact PPG (cPPG) signals using available video dataset and (ii) estimate BP from converted cPPG signals by exploiting large existing databases (e.g. MIMIC). This article presents the first developments where a method for converting iPPG signals measured using a camera into cPPG signals measured by contact sensors is proposed. Real and imaginary parts of the continuous wavelet transform (CWT) of cPPG and iPPG signals are passed to various deep pre-trained U-shaped architectures. Conventional metrics and specific waveform estimators have been implemented to validate the relevance of the predictions. The results exhibit good agreements towards a large portion of metrics, showing that the neural architectures properly estimated cPPG from iPPG signals through their CWT representations. The performance indicates that BP estimation from iPPG signals converted to cPPG signals can now be envisaged. Consequently, future work will focus on the integration of models dedicated to BP estimation trained on MIMIC. This is the first demonstration of a method for accurate reconstruction of cPPG from iPPG signals satisfying pulse waveform criteria.

Affective State during Physiotherapy and Its Analysis Using Machine Learning Methods

Invasive or uncomfortable procedures especially during healthcare trigger emotions. Technological development of the equipment and systems for monitoring and recording psychophysiological functions enables continuous observation of changes to a situation responding to a situation. The presented study aimed to focus on the analysis of the individual’s affective state. The results reflect the excitation expressed by the subjects’ statements collected with psychological questionnaires. The research group consisted of 49 participants (22 women and 25 men). The measurement protocol included acquiring the electrodermal activity signal, cardiac signals, and accelerometric signals in three axes. Subjective measurements were acquired for affective state using the JAWS questionnaires, for cognitive skills the DST, and for verbal fluency the VFT. The physiological and psychological data were subjected to statistical analysis and then to a machine learning process using different features selection methods (JMI or PCA). The highest accuracy of the kNN classifier was achieved in combination with the JMI method (81.63%) concerning the division complying with the JAWS test results. The classification sensitivity and specificity were 85.71% and 71.43%.