Heart Rate and Heart Rate Variability From Single-Channel Video and ICA Integration of Multiple Signals

Deep learning and remote photoplethysmography powered advancements in contactless physiological measurement

In recent decades, there has been ongoing development in the application of computer vision (CV) in the medical field. As conventional contact-based physiological measurement techniques often restrict a patient's mobility in the clinical environment, the ability to achieve continuous, comfortable and convenient monitoring is thus a topic of interest to researchers. One type of CV application is remote imaging photoplethysmography (rPPG), which can predict vital signs using a video or image. While contactless physiological measurement techniques have an excellent application prospect, the lack of uniformity or standardization of contactless vital monitoring methods limits their application in remote healthcare/telehealth settings. Several methods have been developed to improve this limitation and solve the heterogeneity of video signals caused by movement, lighting, and equipment. The fundamental algorithms include traditional algorithms with optimization and developing deep learning (DL) algorithms. This article aims to provide an in-depth review of current Artificial Intelligence (AI) methods using CV and DL in contactless physiological measurement and a comprehensive summary of the latest development of contactless measurement techniques for skin perfusion, respiratory rate, blood oxygen saturation, heart rate, heart rate variability, and blood pressure.

Evaluation of transformation invariant loss function with distance equilibrium in prediction of imaging photoplethysmography characteristics

Objective. Monitoring changes in human heart rate variability (HRV) holds significant importance for protecting life and health. Studies have shown that Imaging Photoplethysmography (IPPG) based on ordinary color cameras can detect the color change of the skin pixel caused by cardiopulmonary system. Most researchers employed deep learning IPPG algorithms to extract the blood volume pulse (BVP) signal, analyzing it predominantly through the heart rate (HR). However, this approach often overlooks the inherent intricate time-frequency domain characteristics in the BVP signal, which cannot be comprehensively deduced solely from HR. The analysis of HRV metrics through the BVP signal is imperative.

APPROACH

In this paper, the transformation invariant loss function with distance equilibrium (TIDLE) loss function is applied to IPPG for the first time, and the details of BVP signal can be recovered better. In detail, TIDLE is tested in four commonly used IPPG deep learning models, which are DeepPhys, EfficientPhys, Physnet and TS_CAN, and compared with other three loss functions, which are mean absolute error (MAE), mean square error (MSE), Neg Pearson Coefficient correlation (NPCC).

MAIN RESULTS

The experiments demonstrate that MAE and MSE exhibit suboptimal performance in predicting LF/HF across the four models, achieving the Statistic of Mean Absolute Error (MAES) of 25.94% and 34.05%, respectively. In contrast, NPCC and TIDLE yielded more favorable results at 13.51% and 11.35%, respectively. Taking into consideration the morphological characteristics of the BVP signal, on the two optimal models for predicting HRV metrics, namely DeepPhys and TS_CAN, the Pearson coefficients for the BVP signals predicted by TIDLE in comparison to the gold-standard BVP signals achieved values of 0.627 and 0.605, respectively. In contrast, the results based on NPCC were notably lower, at only 0.545 and 0.533, respectively.

SIGNIFICANCE

This paper contributes significantly to the effective restoration of the morphology and frequency domain characteristics of the BVP signal.

DEMA: A Deep Learning-Enabled Model for Non-Invasive Human Vital Signs Monitoring Based on Optical Fiber Sensing

Optical fiber sensors are extensively employed for their unique merits, such as small size, being lightweight, and having strong robustness to electronic interference. The above-mentioned sensors apply to more applications, especially the detection and monitoring of vital signs in medical or clinical. However, it is inconvenient for daily long-term human vital sign monitoring with conventional monitoring methods under the uncomfortable feelings generated since the skin and devices come into direct contact. This study introduces a non-invasive surveillance system that employs an optical fiber sensor and advanced deep-learning methodologies for precise vital sign readings. This system integrates a monitor based on the MZI (Mach-Zehnder interferometer) with LSTM networks, surpassing conventional approaches and providing potential uses in medical diagnostics. This could be potentially utilized in non-invasive health surveillance, evaluation, and intelligent health care.

Uncertainties in the Analysis of Heart Rate Variability: A Systematic Review

Heart rate variability (HRV) is an important metric with a variety of applications in clinical situations such as cardiovascular diseases, diabetes mellitus, and mental health. HRV data can be potentially obtained from electrocardiography and photoplethysmography signals, then computational techniques such as signal filtering and data segmentation are used to process the sampled data for calculating HRV measures. However, uncertainties arising from data acquisition, computational models, and physiological factors can lead to degraded signal quality and affect HRV analysis. Therefore, it is crucial to address these uncertainties and develop advanced models for HRV analysis. Although several reviews of HRV analysis exist, they primarily focus on clinical applications, trends in HRV methods, or specific aspects of uncertainties such as measurement noise. This paper provides a comprehensive review of uncertainties in HRV analysis, quantifies their impacts, and outlines potential solutions. To the best of our knowledge, this is the first study that presents a holistic review of uncertainties in HRV methods and quantifies their impacts on HRV measures from an engineer's perspective. This review is essential for developing robust and reliable models, and could serve as a valuable future reference in the field, particularly for dealing with uncertainties in HRV analysis.

Research on heart rate extraction method based on mobile phone video

As an important indicator of human health, heart rate is related to the diagnosis of cardiovascular diseases. In recent years, extracting the heart rate from the mobile phone image has become a research hotspot. However, the illumination intensity of the background, frame rate of the video, and resolution of the image influence heart rate detection accuracy. To overcome these problems, this study proposed a novel heart rate extraction method based on mobile video. Firstly, the mobile phone camera is engaged to record the finger video, the region of interest (ROI) is extracted through the iterative threshold, and the pulse signal is obtained according to the grayscale change of the resolution within the ROI. Then, a low-pass and a high-pass Butterworth filters are exploited to filter out the noise and interframes from the extracted pulse signal. Finally, an improved adaptive peak extraction algorithm is proposed to detect the pulse peaks and the heart rate derived from the difference in pulse peaks. The experimental results show that light intensity, frame rate and resolution all have an influence on the heart rate extraction accuracy, with the most obvious influence of light, the average accuracy of the experiment can reach 99.32 % under good lighting conditions, while only 72.23 % under poor lighting conditions. In terms of frame rate, increasing the frame rate from 30 fps to 60 fps, the accuracy is improved by 0.9 %. For the resolution, increasing the resolution from 1080 p to 2160 p, the accuracy is improved by 1.12 %. While comparing the proposed method with existing methods, the proposed method has a higher accuracy rate, which has important practical value and application prospects in telemedicine and daily monitoring.

Prediction of Impulsive Aggression Based on Video Images

In response to the subjectivity, low accuracy, and high concealment of existing attack behavior prediction methods, a video-based impulsive aggression prediction method that integrates physiological parameters and facial expression information is proposed. This method uses imaging equipment to capture video and facial expression information containing the subject's face and uses imaging photoplethysmography (IPPG) technology to obtain the subject's heart rate variability parameters. Meanwhile, the ResNet-34 expression recognition model was constructed to obtain the subject's facial expression information. Based on the random forest classification model, the physiological parameters and facial expression information obtained are used to predict individual impulsive aggression. Finally, an impulsive aggression induction experiment was designed to verify the method. The experimental results show that the accuracy of this method for predicting the presence or absence of impulsive aggression was 89.39%. This method proves the feasibility of applying physiological parameters and facial expression information to predict impulsive aggression. This article has important theoretical and practical value for exploring new impulsive aggression prediction methods. It also has significance in safety monitoring in special and public places such as prisons and rehabilitation centers.

Contactless Cardiovascular Assessment by Imaging Photoplethysmography: A Comparison with Wearable Monitoring

Despite the notable recent developments in the field of remote photoplethysmography (rPPG), extracting a reliable pulse rate variability (PRV) signal still remains a challenge. In this study, eight image-based photoplethysmography (iPPG) extraction methods (GRD, AGRD, PCA, ICA, LE, SPE, CHROM, and POS) were compared in terms of pulse rate (PR) and PRV features. The algorithms were made robust for motion and illumination artifacts by using ad hoc pre- and postprocessing steps. Then, they were systematically tested on the public dataset UBFC-RPPG, containing data from 42 subjects sitting in front of a webcam (30 fps) while playing a time-sensitive mathematical game. The performances of the algorithms were evaluated by statistically comparing iPPG-based and finger-PPG-based PR and PRV features in terms of Spearman's correlation coefficient, normalized root mean square error (NRMSE), and Bland-Altman analysis. The study revealed POS and CHROM techniques to be the most robust for PR estimation and the assessment of overall autonomic nervous system (ANS) dynamics by using PRV features in time and frequency domains. Furthermore, we demonstrated that a reliable characterization of the vagal tone is made possible by computing the Poincaré map of PRV series derived from the POS and CHROM methods. This study supports the use of iPPG systems as promising tools to obtain clinically useful and specific information about ANS dynamics.

Unobtrusive Heart Rate Monitoring using Near-Infrared Imaging During Driving

In-vehicle health monitoring allows for continuous vital sign measurement in everyday life. Eventually, this could lead to early detection of cardiovascular diseases. In this work, we propose non-contact heart rate (HR) monitoring utilizing near-infrared (NIR) camera technology. Ten healthy volunteers are monitored in a realistic driving simulator during resting (5 min) and driving (10 min). We synchronously acquire videos using an out-of-the-shelf, low-cost NIR camera and 3-lead electrocardiography (ECG) serves as ground truth. The MediaPipe face detector delivers the region of interest (ROI) and we determine the HR from the peak with maximum amplitude within the power spectrum of skin color changes. We compare video-based with ECG-based HR, resulting in a mean absolute error (MAE) of 7.8 bpm and 13.0 bpm in resting and driving condition, respectively. As we apply only a simple signal processing pipeline without sophisticated filtering, we conclude that NIR camera-based HR measurements enables unobtrusive and non-contact monitoring to a certain extent, but artifacts from subject movement pose a challenge. If these issues can be addressed, continuous vital sign measurement in everyday life could become reality.

A deep learning approach to estimate pulse rate by remote photoplethysmography

OBJECTIVE

This study proposes an U-net shaped Deep Neural Network (DNN) model to extract remote photoplethysmography (rPPG) signals from skin color signals to estimate Pulse Rate (PR).

APPROACH

Three input window sizes are used into the DNN: 256 samples (5.12 s), 512 samples (10.24 s), and 1024 (20.48 s). A data argumentation algorithm based on interpolation is also used here to artificially increase the number of training samples.

MAIN RESULTS

The proposed model outperformed a prior-knowledge rPPG method by using input signals with window of 256 and 512 samples. Also, it was found that the data augmentation procedure only increased the performance for window of 1024 samples. The trained model achieved a Mean Absolute Error (MAE) of 3.97 Beats per Minute (BPM) and Root Mean Squared Error (RMSE) of 6.47 BPM, for the 256 samples window, and MAE of 3.00 BPM and RMSE of 5.45 BPM for the window of 512 samples. On the other hand, the prior-knowledge rPPG method got a MAE of 8.04 BPM and RMSE of 16.63 BPM for the window of 256 samples, and MAE of 3.49 BPM and RMSE of 7.92 BPM for the window of 512. For the longest window (1024 samples), the concordance of the predicted PRs from the DNNs and the true PRs was higher when applying the data augmentation procedure.

SIGNIFICANCE

These results demonstrate a big potential of this technique for PR estimation, showing that the DNN proposed here may generate reliable rPPG signals even with short window lengths (5.12 s and 10.24 s), suggesting that it needs less data for a faster rPPG measurement and PR estimation.

Contactless Vital Signs Monitoring From Videos Recorded With Digital Cameras: An Overview

The measurement of physiological parameters is fundamental to assess the health status of an individual. The contactless monitoring of vital signs may provide benefits in various fields of application, from healthcare and clinical setting to occupational and sports scenarios. Recent research has been focused on the potentiality of camera-based systems working in the visible range (380-750 nm) for estimating vital signs by capturing subtle color changes or motions caused by physiological activities but invisible to human eyes. These quantities are typically extracted from videos framing some exposed body areas (e.g., face, torso, and hands) with adequate post-processing algorithms. In this review, we provided an overview of the physiological and technical aspects behind the estimation of vital signs like respiratory rate, heart rate, blood oxygen saturation, and blood pressure from digital images as well as the potential fields of application of these technologies. Per each vital sign, we provided the rationale for the measurement, a classification of the different techniques implemented for post-processing the original videos, and the main results obtained during various applications or in validation studies. The available evidence supports the premise of digital cameras as an unobtrusive and easy-to-use technology for physiological signs monitoring. Further research is needed to promote the advancements of the technology, allowing its application in a wide range of population and everyday life, fostering a biometrical holistic of the human body (BHOHB) approach.

A novel fetal ecg signal extraction from maternal ecg signal using conditional generative adversarial networks (CGAN)

Fetal Electrocardiogram (ECG) signal extraction from non-invasive abdominal ECG signal is one of the important clinical practices followed to observe the fetal health state. Information about heart growth and health conditions of a fetus can be observed from fetal ECG signals. However, acquiring fetal ECG from abdominal ECG signals is still considered as a challenging task in biomedical analysis. This is mainly due to corrupted high amplitude maternal ECG signals, low signal to noise ratio of fetal ECG signal, difficulties in reduction of QRS (Q wave, R wave, S wave) complexities, fetal ECG signal superimposed characteristics, other motion, and electromyography artifacts. To reduce these conventional challenges, in fetal ECG analysis of a novel Conditional Generative adversarial network (CGAN) is introduced in this research work to extract the fetal ECG signal. The proposed classification model was classified efficiently in fetal ECG signals from non-invasive abdominal ECG signals. The experimental analysis demonstrates that the proposed network model provides better results in terms of sensitivity, specificity, and accuracy compared to the conventional fetal ECG extraction models like singular value decomposition, periodic component analysis, and Adaptive neuro-fuzzy inference system.

VidAF: A Motion-Robust Model for Screening Atrial Fibrillation from Facial Videos

Atrial fibrillation (AF) is the most common arrhythmia, but an estimated 30% of patients with AF are unaware of their conditions. The purpose of this work is to design a model for AF screening from facial videos, with a focus on addressing typical motion disturbances in our real life, such as head movements and expression changes. This model detects a pulse signal from the skin color changes in a facial video by a convolution neural network, incorporating a phase-driven attention mechanism to suppress motion signals in the space domain. It then encodes the pulse signal into discriminative features for AF classification by a coding neural network, using a de-noise coding strategy to improve the robustness of the features to motion signals in the time domain. The proposed model was tested on a dataset containing 1200 samples of 100 AF patients and 100 non-AF subjects. Experimental results demonstrated that VidAF had significant robustness to facial motions, predicting clean pulse signals with the mean absolute error of inter-pulse intervals less than 100 milliseconds. Besides, the model achieved promising performance in AF identification, showing an accuracy of more than 90% in multiple challenging scenarios. VidAF provides a more convenient and cost-effective approach for opportunistic AF screening in the community.

Recent Advances in Atherosclerotic Disease Screening Using Pervasive Healthcare

Atherosclerosis screening helps the medical model transform from therapeutic medicine to preventive medicine by assessing degree of atherosclerosis prior to the occurrence of fatal vascular events. Pervasive screening emphasizes atherosclerotic monitoring with easy access, quick process, and advanced computing. In this work, we introduced five cutting-edge pervasive technologies including imaging photoplethysmography (iPPG), laser Doppler, radio frequency (RF), thermal imaging (TI), optical fiber sensing and piezoelectric sensor. IPPG measures physiological parameters by using video images that record the subtle skin color changes consistent with cardiac-synchronous blood volume changes in subcutaneous arteries and capillaries. Laser Doppler obtained the information on blood flow by analyzing the spectral components of backscattered light from the illuminated tissues surface. RF is based on Doppler shift caused by the periodic movement of the chest wall induced by respiration and heartbeat. TI measures vital signs by detecting electromagnetic radiation emitted by blood flow. The working principle of optical fiber sensor is to detect the change of light properties caused by the interaction between the measured physiological parameter and the entering light. Piezoelectric sensors are based on the piezoelectric effect of dielectrics. All these pervasive technologies are noninvasive, mobile, and can detect physiological parameters related to atherosclerosis screening.

Modulation Model of the Photoplethysmography Signal for Vital Sign Extraction

This paper introduces an amplitude and frequency modulation (AM-FM) model to characterize the photoplethysmography (PPG) signal. The model indicates that the PPG signal spectrum contains one dominant frequency component - the heart rate (HR), which is guarded by two weaker frequency components on both sides; the distance from the dominant component to the guard components represents the respiratory rate (RR). Based on this model, an efficient algorithm is proposed to estimate both HR and RR by searching for the dominant frequency component and two guard components. The proposed method is performed in the frequency domain to estimate RR, which is more robust to additive noise than the prior art based on temporal features. Experiments were conducted on two types of PPG signals collected with a contact sensor (an oximeter) and a contactless visible imaging sensor (a color camera), respectively. The PPG signal from the contactless sensor is much noisier than the signal from the contact sensor. The experimental results demonstrate the effectiveness of the proposed algorithm, including under relatively noisy scenarios.