Automatic Selection of Webcam Photoplethysmographic Pixels Based on Lightness Criteria

Determine atrial fibrillation burden with a photoplethysmographic mobile sensor: the atrial fibrillation burden trial: detection and quantification of episodes of atrial fibrillation using a cloud analytics service connected to a wearable with photoplethysmographic sensor

Aims

Recent studies suggest that atrial fibrillation (AF) burden (time AF is present) is an independent risk factor for stroke. The aim of this trial was to study the feasibility and accuracy to identify AF episodes and quantify AF burden in patients with a known history of paroxysmal AF with a photoplethysmography (PPG)-based wearable.

Methods and results

In this prospective, single-centre trial, the PPG-based estimation of AF burden was compared with measurements of a conventional 48 h Holter electrocardiogram (ECG), which served as the gold standard. An automated algorithm performed PPG analysis, while a cardiologist, blinded for the PPG data, analysed the ECG data. Detected episodes of AF measured by both methods were aligned timewise.Out of 100 patients recruited, 8 had to be excluded due to technical issues. Data from 92 patients were analysed [55.4% male; age 73.3 years (standard deviation, SD: 10.4)]. Twenty-five patients presented AF during the study period. The intraclass correlation coefficient of total AF burden minutes detected by the two measurement methods was 0.88. The percentage of correctly identified AF burden over all patients was 85.1% and the respective parameter for non-AF time was 99.9%.

Conclusion

Our results demonstrate that a PPG-based wearable in combination with an analytical algorithm appears to be suitable for a semiquantitative estimation of AF burden in patients with a known history of paroxysmal AF.

Trial Registration number

NCT04563572.

X-iPPGNet: A novel one stage deep learning architecture based on depthwise separable convolutions for video-based pulse rate estimation

Pulse rate (PR) is one of the most important markers for assessing a person's health. With the increasing demand for long-term health monitoring, much attention is being paid to contactless PR estimation using imaging photoplethysmography (iPPG). This non-invasive technique is based on the analysis of subtle changes in skin color. Despite efforts to improve iPPG, the existing algorithms are vulnerable to less-constrained scenarios (i.e., head movements, facial expressions, and environmental conditions). In this article, we propose a novel end-to-end spatio-temporal network, namely X-iPPGNet, for instantaneous PR estimation directly from facial video recordings. Unlike most existing systems, our model learns the iPPG concept from scratch without incorporating any prior knowledge or going through the extraction of blood volume pulse signals. Inspired by the Xception network architecture, color channel decoupling is used to learn additional photoplethysmographic information and to effectively reduce the computational cost and memory requirements. Moreover, X-iPPGNet predicts the pulse rate from a short time window (2 s), which has advantages with high and sharply fluctuating pulse rates. The experimental results revealed high performance under all conditions including head motions, facial expressions, and skin tone. Our approach significantly outperforms all current state-of-the-art methods on three benchmark datasets: MMSE-HR (MAE = 4.10 ; RMSE = 5.32 ; r = 0.85), UBFC-rPPG (MAE = 4.99 ; RMSE = 6.26 ; r = 0.67), MAHNOB-HCI (MAE = 3.17 ; RMSE = 3.93 ; r = 0.88).

Photoplethysmographic Imaging of Hemodynamics and Two-Dimensional Oximetry

The review of recent papers devoted to actively developing methods of photoplethysmographic imaging (the PPGI) of blood volume pulsations in vessels and non-contact two-dimensional oximetry on the surface of a human body has been carried out. The physical fundamentals and technical aspects of the PPGI and oximetry have been considered. The manifold of the physiological parameters available for the analysis by the PPGI method has been shown. The prospects of the PPGI technology have been discussed. The possibilities of non-contact determination of blood oxygen saturation SpO2 (pulse saturation O2) have been described. The relevance of remote determination of the level of oxygenation in connection with the spread of a new coronavirus infection SARS-CoV-2 (COVID-19) has been emphasized. Most of the works under consideration cover the period 2010-2021.

Real-time realizable mobile imaging photoplethysmography

Photoplethysmography imaging (PPGI) sensors have attracted a significant amount of attention as they enable the remote monitoring of heart rates (HRs) and thus do not require any additional devices to be worn on fingers or wrists. In this study, we mounted PPGI sensors on a robot for active and autonomous HR (R-AAH) estimation. We proposed an algorithm that provides accurate HR estimation, which can be performed in real time using vision and robot manipulation algorithms. By simplifying the extraction of facial skin images using saturation (S) values in the HSV color space, and selecting pixels based on the most frequent S value within the face image, we achieved a reliable HR assessment. The results of the proposed algorithm using the R-AAH method were evaluated by rigorous comparison with the results of existing algorithms on the UBFC-RPPG dataset (n = 42). The proposed algorithm yielded an average absolute error (AAE) of 0.71 beats per minute (bpm). The developed algorithm is simple, with a processing time of less than 1 s (275 ms for an 8-s window). The algorithm was further validated on our own dataset (BAMI-RPPG dataset [n = 14]) with an AAE of 0.82 bpm.

iPPG 2 cPPG: Reconstructing contact from imaging photoplethysmographic signals using U-Net architectures

Imaging photoplethysmography (iPPG) is an optical technique dedicated to the assessment of several vital functions using a simple camera. Significant efforts have been made to reliably estimate heart and respiratory rates. Currently, research is focusing on the remote estimation of oxygen saturation and blood pressure (BP). The limited number of publicly available data tends to restrict the advancements related to BP estimation. To overcome this limit, we propose to split the problem in a two-stage processing chain: (i) converting iPPG to contact PPG (cPPG) signals using available video dataset and (ii) estimate BP from converted cPPG signals by exploiting large existing databases (e.g. MIMIC). This article presents the first developments where a method for converting iPPG signals measured using a camera into cPPG signals measured by contact sensors is proposed. Real and imaginary parts of the continuous wavelet transform (CWT) of cPPG and iPPG signals are passed to various deep pre-trained U-shaped architectures. Conventional metrics and specific waveform estimators have been implemented to validate the relevance of the predictions. The results exhibit good agreements towards a large portion of metrics, showing that the neural architectures properly estimated cPPG from iPPG signals through their CWT representations. The performance indicates that BP estimation from iPPG signals converted to cPPG signals can now be envisaged. Consequently, future work will focus on the integration of models dedicated to BP estimation trained on MIMIC. This is the first demonstration of a method for accurate reconstruction of cPPG from iPPG signals satisfying pulse waveform criteria.

Adaptive Gaussian Mixture Model Driven Level Set Segmentation for Remote Pulse Rate Detection

This paper presents an approach for pulse rate extraction from videos. The core of the presented approach is a novel method to segment and track a suitable region of interest (ROI). The proposed method combines level sets with subject-individual Gaussian Mixture Models to yield a time varying ROI. The ROI builds up from multiple homogeneous skin areas under constraints regarding the area and contour length of the ROI. Together with state of the art signal processing methods our approach yields an Mean Average Error (MAE) of 2.3 bpm, 1.4 bpm and 2.7 bpm on own data, the PURE database and the UBFC-rPPG database, respectively. Therewith, our method performs equal or better compared to widely used approaches (e.g. the KLT tracker instead of the proposed image processing yields an MAE of 2.6 bpm, 2.6 bpm and 4.4 bpm). Such results and the 2nd place with a MAE of 7.92 bpm in the 1st Challenge on Remote Physiological Signal Sensing prove the applicability of the proposed method. The taken approach, however, bears further potential for optimization in the context of photoplethysmography imaging and should be transferable to other segmentation tasks as well.

Is Continuous Heart Rate Monitoring of Livestock a Dream or Is It Realistic? A Review

For all homoeothermic living organisms, heart rate (HR) is a core variable to control the metabolic energy production in the body, which is crucial to realize essential bodily functions. Consequently, HR monitoring is becoming increasingly important in research of farm animals, not only for production efficiency, but also for animal welfare. Real-time HR monitoring for humans has become feasible though there are still shortcomings for continuously accurate measuring. This paper is an effort to estimate whether it is realistic to get a continuous HR sensor for livestock that can be used for long term monitoring. The review provides the reported techniques to monitor HR of living organisms by emphasizing their principles, advantages, and drawbacks. Various properties and capabilities of these techniques are compared to check the potential to transfer the mostly adequate sensor technology of humans to livestock in term of application. Based upon this review, we conclude that the photoplethysmographic (PPG) technique seems feasible for implementation in livestock. Therefore, we present the contributions to overcome challenges to evolve to better solutions. Our study indicates that it is realistic today to develop a PPG sensor able to be integrated into an ear tag for mid-sized and larger farm animals for continuously and accurately monitoring their HRs.

3D Convolutional Neural Networks for Remote Pulse Rate Measurement and Mapping from Facial Video

Remote pulse rate measurement from facial video has gained particular attention over the last few years. Research exhibits significant advancements and demonstrates that common video cameras correspond to reliable devices that can be employed to measure a large set of biomedical parameters without any contact with the subject. A new framework for measuring and mapping pulse rate from video is presented in this pilot study. The method, which relies on convolutional 3D networks, is fully automatic and does not require any special image preprocessing. In addition, the network ensures concurrent mapping by producing a prediction for each local group of pixels. A particular training procedure that employs only synthetic data is proposed. Preliminary results demonstrate that this convolutional 3D network can effectively extract pulse rate from video without the need for any processing of frames. The trained model was compared with other state-of-the-art methods on public data. Results exhibit significant agreement between estimated and ground-truth measurements: the root mean square error computed from pulse rate values assessed with the convolutional 3D network is equal to 8.64 bpm, which is superior to 10 bpm for the other state-of-the-art methods. The robustness of the method to natural motion and increases in performance correspond to the two main avenues that will be considered in future works.

Cardiovascular assessment by imaging photoplethysmography - a review

Over the last few years, the contactless acquisition of cardiovascular parameters using cameras has gained immense attention. The technique provides an optical means to acquire cardiovascular information in a very convenient way. This review provides an overview on the technique's background and current realizations. Besides giving detailed information on the most widespread application of the technique, namely the contactless acquisition of heart rate, we outline further concepts and we critically discuss the current state.

Using imaging photoplethysmography for heart rate estimation in non-human primates

For humans and for non-human primates heart rate is a reliable indicator of an individual's current physiological state, with applications ranging from health checks to experimental studies of cognitive and emotional state. In humans, changes in the optical properties of the skin tissue correlated with cardiac cycles (imaging photoplethysmogram, iPPG) allow non-contact estimation of heart rate by its proxy, pulse rate. Yet, there is no established simple and non-invasive technique for pulse rate measurements in awake and behaving animals. Using iPPG, we here demonstrate that pulse rate in rhesus monkeys can be accurately estimated from facial videos. We computed iPPGs from eight color facial videos of four awake head-stabilized rhesus monkeys. Pulse rate estimated from iPPGs was in good agreement with reference data from a contact pulse-oximeter: the error of pulse rate estimation was below 5% of the individual average pulse rate in 83% of the epochs; the error was below 10% for 98% of the epochs. We conclude that iPPG allows non-invasive and non-contact estimation of pulse rate in non-human primates, which is useful for physiological studies and can be used toward welfare-assessment of non-human primates in research.

Lock-in technique for extraction of pulse rates and associated confidence levels from video

We investigate the practical applicability of video photoplethysmography (VPPG) to extract heart rates of subjects using noncontact color video recordings of human faces collected under typical indoor laboratory conditions using commercial video cameras. Videos were processed following three previously described simple VPPG algorithms to produce a time-varying plethysmographic signal. These time signals were then analyzed using, to the best of our knowledge, a novel, lock-in algorithm that was developed to extract the pulsatile frequency component. A protocol to associate confidence estimates for the extracted heart rates for each video stream is presented. Results indicate that the difference between heart rates extracted using the lock-in technique and gold-standard measurements, for videos with high-confidence metrics, was less than 4 beats per minute. Constraints on video acquisition and processing, including natural subject motion and the total duration of video recorded required for evaluating these confidence metrics, are discussed.

Camera-based photoplethysmography in an intraoperative setting

Background

Camera-based photoplethysmography (cbPPG) is a measurement technique which enables remote vital sign monitoring by using cameras. To obtain valid plethysmograms, proper regions of interest (ROIs) have to be selected in the video data. Most automated selection methods rely on specific spatial or temporal features limiting a broader application. In this work, we present a new method which overcomes those drawbacks and, therefore, allows cbPPG to be applied in an intraoperative environment.

Methods

We recorded 41 patients during surgery using an RGB and a near-infrared (NIR) camera. A Bayesian skin classifier was employed to detect suitable regions, and a level set segmentation approach to define and track ROIs based on spatial homogeneity.

Results

The results show stable and homogeneously illuminated ROIs. We further evaluated their quality with regards to extracted cbPPG signals. The green channel provided the best results where heart rates could be correctly estimated in 95.6% of cases. The NIR channel yielded the highest contribution in compensating false estimations.

Conclusions

The proposed method proved that cbPPG is applicable in intraoperative environments. It can be easily transferred to other settings regardless of which body site is considered.

Electronic supplementary material

The online version of this article (10.1186/s12938-018-0467-7) contains supplementary material, which is available to authorized users.