Motion robust PPG-imaging through color channel mapping

Multispectral Depolarization Based Living-Skin Detection: A New Measurement Principle

Camera-based photoplethysmographic imaging enabled the segmentation of living-skin tissues in a video, but it has inherent limitations to be used in real-life applications such as video health monitoring and face anti-spoofing. Inspired by the use of polarization for improving vital signs monitoring (i.e. specular reflection removal), we observed that skin tissues have an attractive property of wavelength-dependent depolarization due to its multi-layer structure containing different absorbing chromophores, i.e. polarized light photons with longer wavelengths (R) have deeper skin penetrability and thus experience thorougher depolarization than those with shorter wavelengths (G and B). Thus we proposed a novel dual-polarization setup and an elegant algorithm (named "MSD") that exploits the nature of multispectral depolarization of skin tissues to detect living-skin pixels, which only requires two images sampled at the parallel and cross polarizations to estimate the characteristic chromaticity changes (R/G) caused by tissue depolarization. Our proposal was verified in both the laboratory and hospital settings (ICU and NICU) focused on anti-spoofing and patient skin segmentation. The clinical experiments in ICU also indicate the potential of MSD for skin perfusion analysis, which may lead to a new diagnostic imaging approach in the future.

Pulse Oximetry Imaging System Using Spatially Uniform Dual Wavelength Illumination

Pulse oximetry is a non-invasive method for measuring blood oxygen saturation. However, its detection scheme heavily relies on single-point measurements. If the oxygen saturation is measured at a single location, the measurements are influenced by the profile of illumination, spatial variations in blood flow, and skin pigment. To overcome these issues, imaging systems that measure the distribution of oxygen saturation have been demonstrated. However, previous imaging systems have relied on red and near-infrared illuminations with different profiles, resulting in inconsistent ratios between transmitted red and near-infrared light over space. Such inconsistent ratios can introduce fundamental errors when calculating the spatial distribution of oxygen saturation. In this study, we developed a novel illumination system specifically designed for a pulse oximetry imaging system. For the illumination system, we customized the integrating sphere by coating a mixture of barium sulfate and white paint inside it and by coupling eight red and eight near-infrared LEDs. The illumination system created identical patterns of red and near-infrared illuminations that were spatially uniform. This allowed the ratio between transmitted red and near-infrared light to be consistent over space, enabling the calculation of the spatial distribution of oxygen saturation. We believe our developed pulse oximetry imaging system can be used to obtain spatial information on blood oxygen saturation that provides insight into the oxygenation of the blood contained within the peripheral region of the tissue.

X-iPPGNet: A novel one stage deep learning architecture based on depthwise separable convolutions for video-based pulse rate estimation

Pulse rate (PR) is one of the most important markers for assessing a person's health. With the increasing demand for long-term health monitoring, much attention is being paid to contactless PR estimation using imaging photoplethysmography (iPPG). This non-invasive technique is based on the analysis of subtle changes in skin color. Despite efforts to improve iPPG, the existing algorithms are vulnerable to less-constrained scenarios (i.e., head movements, facial expressions, and environmental conditions). In this article, we propose a novel end-to-end spatio-temporal network, namely X-iPPGNet, for instantaneous PR estimation directly from facial video recordings. Unlike most existing systems, our model learns the iPPG concept from scratch without incorporating any prior knowledge or going through the extraction of blood volume pulse signals. Inspired by the Xception network architecture, color channel decoupling is used to learn additional photoplethysmographic information and to effectively reduce the computational cost and memory requirements. Moreover, X-iPPGNet predicts the pulse rate from a short time window (2 s), which has advantages with high and sharply fluctuating pulse rates. The experimental results revealed high performance under all conditions including head motions, facial expressions, and skin tone. Our approach significantly outperforms all current state-of-the-art methods on three benchmark datasets: MMSE-HR (MAE = 4.10 ; RMSE = 5.32 ; r = 0.85), UBFC-rPPG (MAE = 4.99 ; RMSE = 6.26 ; r = 0.67), MAHNOB-HCI (MAE = 3.17 ; RMSE = 3.93 ; r = 0.88).

Photoplethysmographic Imaging of Hemodynamics and Two-Dimensional Oximetry

The review of recent papers devoted to actively developing methods of photoplethysmographic imaging (the PPGI) of blood volume pulsations in vessels and non-contact two-dimensional oximetry on the surface of a human body has been carried out. The physical fundamentals and technical aspects of the PPGI and oximetry have been considered. The manifold of the physiological parameters available for the analysis by the PPGI method has been shown. The prospects of the PPGI technology have been discussed. The possibilities of non-contact determination of blood oxygen saturation SpO2 (pulse saturation O2) have been described. The relevance of remote determination of the level of oxygenation in connection with the spread of a new coronavirus infection SARS-CoV-2 (COVID-19) has been emphasized. Most of the works under consideration cover the period 2010-2021.

Imaging photoplethysmography as an easy-to-use tool for monitoring changes in tissue blood perfusion during abdominal surgery

Evaluation of tissue perfusion at various stages of surgery is of great importance for the implementation of the concept of safe surgery, including operations on the abdominal organs. Currently, there is no accurate and reliable intraoperative method for assessing tissue perfusion that could help surgeons determine the risks of ischemia and improve outcomes. We propose novel method of intraoperative assessment of tissue perfusion using video camera synchronized with the electrocardiogram. The technique is referred to as imaging photoplethysmography (iPPG). It can be used continuously for monitoring blood supply to organs e.g., before and after anastomosis. In our study, we followed 14 different surgical cases (four stomach and ten colorectal cancers) requiring reconstruction of various organs with anastomosis. With iPPG, intraoperative blood perfusion was successfully visualized and quantified in all 14 patients under study. As most indicative, here we describe in detail two clinical demonstrations during gastrectomy for gastric cancer and right-sided hemicolectomy for cancer of the ascending colon. Feasibility of the iPPG system to assess blood perfusion in organs before and after anastomosis during open surgery was demonstrated for the first time.

Fundamentals of Camera-PPG based Magnetic Resonance Imaging

In Magnetic Resonance Imaging (MRI), cardiac triggering that synchronizes data acquisition with cardiac contractions is an essential technique for acquiring high-quality images. Triggering is typically based on the Electrocardiogram (ECG) signal (e.g. R-peak). Since ECG acquisition involves extra workflow steps like electrode placement and ECG signals are usually disturbed by magnetic fields in high Magnetic Resonance (MR) systems, we explored camera-based photoplethysmography (PPG) as an alternative. We used the in-bore camera of a clinical MR system to investigate the feasibility and challenges of camera-based cardiac triggering. Data from ECG, finger oximeter and camera were synchronously collected. We found that that camera-PPG provides a higher availability of signal (and trigger) measurement, and the PPG signals measured from the forehead show considerably less delay w.r.t. the coupled ECG R-peak than the finger PPG signals in terms of trigger detection. The insights obtained in this study provide a basis for an envisioned system design phase.

Non-contact breathing rate monitoring in newborns: A review

The neonatal period - the first 4 weeks of life - is the most critical time for a child's survival. Breathing rate is a vital indicator of the health condition and requires continuous monitoring in case of sickness or preterm birth. Breathing movements can be counted by contact and non-contact methods. In the case of newborn infants, the non-contact breathing rate monitoring need is high, as a contact-based approach may interfere while providing care and is subject to interference by non-breathing movements. This review article delivers a factual summary, and describes the methods and processing involved in non-contact based breathing rate monitoring. The article also provides the advantages, limitations, and clinical applications of these methods. Additionally, signal processing, feasibility, and future direction of different non-contact neonatal breathing rate monitoring are discussed.

Assessing an Automatic Procedure of Extraction of Physiological Parameters from Skin using Video Photoplethysmography

Video Photoplethysmography (vPPG) allows for estimation of blood volume pulse (BVP) from the skin by means of a video camera recording at high frequency rate. The estimation procedure presents several drawbacks in its application to real world conditions, such as light changes or movements that often generate artifacts in the extracted BVP waveform. In addition, the process requires a skin segmentation algorithm to distinguish skin pixels from the background. To date, even the most refined skin segmentation algorithms still need a manual definition that could lead to incorrect pixel classification, and consequently to a decrease in the signal-to-noise ratio (SNR). We here propose a fully autonomic procedure able to extract BVP from video recordings of the skin in real world conditions.The experimental protocol is designed to record the signals of interest and to evaluate changes in the Autonomic Nervous System modulation of the heart during a baseline condition and a controlled breathing phase. Video recordings are gathered from 4 young healthy subjects (age: 21±1 years). vPPG signals are processed in order to extract the BVP waveform, and a peak detection algorithm detects pulse wave peaks that are then used to compute specific measures of heart rate variability (HRV).The procedure is successfully validated by comparing the extracted HRV measures against those extracted using a finger photoplethysmograph (fPPG) using three different skin segmentation algorithms from BVP signals.

Remote Photoplethysmographic Assessment of the Peripheral Circulation in Critical Care Patients Recovering From Cardiac Surgery

PURPOSE

Camera-based photoplethysmography (cbPPG) remotely detects the volume pulse of cardiac ejection in the peripheral circulation. The cbPPG signal is sourced from the cutaneous microcirculation, yields a 2-dimensional intensity map, and is therefore an interesting monitoring technique. In this study, we investigated whether cbPPG is in general sufficiently sensitive to discern hemodynamic conditions.

METHODS

cbPPG recordings of 70 patients recovering from cardiac surgery were analyzed. Photoplethysmograms were processed offline and the optical pulse power (OPP) of cardiac ejection was calculated. Hemodynamic data, image intensity, and patient movements were recorded synchronously. The effects of hemodynamic parameters and measurement conditions on the patient's individual OPP variability and their actual OPP values were calculated in mixed-effects regression models.

RESULTS

Mean arterial pressure (MAP), pulse pressure (PP), heart rate (HR), and central venous pressure (CVP) significantly explained the individual OPP variability. PP had the highest explanatory power (19.9%). Averaged OPP significantly increased with PP and MAP (P < 0.001, respectively) and decreased with higher HR (P = 0.024). CVP had a 2-directional, nonsignificant effect on averaged OPP. Image intensity and patient movements did significantly affect OPP. After adjustment for hemodynamic covariables and measurement conditions, the effect of PP and HR remained unchanged, whereas that of MAP vanished.

CONCLUSION

cbPPG is sensitive to hemodynamic parameters in critical care patients. It is a potential application for monitoring the peripheral circulation. Its value in a clinical setting has to be determined.

PulseCam: a camera-based, motion-robust and highly sensitive blood perfusion imaging modality

Blood carries oxygen and nutrients to the trillions of cells in our body to sustain vital life processes. Lack of blood perfusion can cause irreversible cell damage. Therefore, blood perfusion measurement has widespread clinical applications. In this paper, we develop PulseCam - a new camera-based, motion-robust, and highly sensitive blood perfusion imaging modality with 1 mm spatial resolution and 1 frame-per-second temporal resolution. Existing camera-only blood perfusion imaging modality suffers from two core challenges: (i) motion artifact, and (ii) small signal recovery in the presence of large surface reflection and measurement noise. PulseCam addresses these challenges by robustly combining the video recording from the camera with a pulse waveform measured using a conventional pulse oximeter to obtain reliable blood perfusion maps in the presence of motion artifacts and outliers in the video recordings. For video stabilization, we adopt a novel brightness-invariant optical flow algorithm that helps us reduce error in blood perfusion estimate below 10% in different motion scenarios compared to 20-30% error when using current approaches. PulseCam can detect subtle changes in blood perfusion below the skin with at least two times better sensitivity, three times better response time, and is significantly cheaper compared to infrared thermography. PulseCam can also detect venous or partial blood flow occlusion that is difficult to identify using existing modalities such as the perfusion index measured using a pulse oximeter. With the help of a pilot clinical study, we also demonstrate that PulseCam is robust and reliable in an operationally challenging surgery room setting. We anticipate that PulseCam will be used both at the bedside as well as a point-of-care blood perfusion imaging device to visualize and analyze blood perfusion in an easy-to-use and cost-effective manner.

An open-source remote heart rate imaging method with practical apparatus and algorithms

Recent developments in computer science and digital image processing have enabled the extraction of an individual’s heart pulsations from pixel changes in recorded video images of human skin surfaces. This method is termed remote photoplethysmography (rPPG) and can be achieved with consumer-level cameras (e.g., a webcam or mobile camera). The goal of the present publication is two-fold. First, we aim to organize future rPPG software developments in a tractable and nontechnical manner, such that the public gains access to a basic open-source rPPG code, comes to understand its utility, and can follow its most recent progressions. The second goal is to investigate rPPG’s accuracy in detecting heart rates from the skin surfaces of several body parts after physical exercise and under ambient lighting conditions with a consumer-level camera. We report that rPPG is highly accurate when the camera is aimed at facial skin tissue, but that the heart rate recordings from wrist regions are less reliable, and recordings from the calves are unreliable. Facial rPPG remained accurate despite the high heart rates after exercise. The proposed research procedures and the experimental findings provide guidelines for future studies on rPPG.

Remote Monitoring of Vital Signs in Diverse Non-Clinical and Clinical Scenarios Using Computer Vision Systems: A Review

Techniques for noncontact measurement of vital signs using camera imaging technologies have been attracting increasing attention. For noncontact physiological assessments, computer vision-based methods appear to be an advantageous approach that could be robust, hygienic, reliable, safe, cost effective and suitable for long distance and long-term monitoring. In addition, video techniques allow measurements from multiple individuals opportunistically and simultaneously in groups. This paper aims to explore the progress of the technology from controlled clinical scenarios with fixed monitoring installations and controlled lighting, towards uncontrolled environments, crowds and moving sensor platforms. We focus on the diversity of applications and scenarios being studied in this topic. From this review it emerges that automatic multiple regions of interest (ROIs) selection, removal of noise artefacts caused by both illumination variations and motion artefacts, simultaneous multiple person monitoring, long distance detection, multi-camera fusion and accepted publicly available datasets are topics that still require research to enable the technology to mature into many real-world applications.

Visualization of skin capillaries with moving red blood cells in arbitrary area of the body

Evaluation of skin microcirculation allows for the assessment of functional states for neuroendocrine and endothelial regulation. We present a novel method to visualize skin microvessels in any area of the body, which is in contrast to classical capillaroscopy, in which the application areas are limited to the nailfold and retina capillaries. The technique is based on microscopic video-image analysis. It exploits a specific feature of irregularity of red-blood-cells motion. Feasibility of the method is demonstrated by mapping the skin capillaries in the forearm and face of 11 healthy volunteers. The proposed method is promising for the quantitative assessment of cutaneous microcirculation in a wide range of diseases and functional states.

A Broader Look: Camera-Based Vital Sign Estimation across the Spectrum

OBJECTIVES

Camera-based vital sign estimation allows the contactless assessment of important physiological parameters. Seminal contributions were made in the 1930s, 1980s, and 2000s, and the speed of development seems ever increasing. In this suivey, we aim to overview the most recent works in this area, describe their common features as well as shortcomings, and highlight interesting "outliers".

METHODS

We performed a comprehensive literature research and quantitative analysis of papers published between 2016 and 2018. Quantitative information about the number of subjects, studies with healthy volunteers vs. pathological conditions, public datasets, laboratory vs. real-world works, types of camera, usage of machine learning, and spectral properties of data was extracted. Moreover, a qualitative analysis of illumination used and recent advantages in terms of algorithmic developments was also performed.

RESULTS

Since 2016, 116 papers were published on camera-based vital sign estimation and 59% of papers presented results on 20 or fewer subjects. While the average number of participants increased from 15.7 in 2016 to 22.9 in 2018, the vast majority of papers (n=100) were on healthy subjects. Four public datasets were used in 10 publications. We found 27 papers whose application scenario could be considered a real-world use case, such as monitoring during exercise or driving. These include 16 papers that dealt with non-healthy subjects. The majority of papers (n=61) presented results based on visual, red-green-blue (RGB) information, followed by RGB combined with other parts of the electromagnetic spectrum (n=18), and thermography only (n=12), while other works (n=25) used other mono- or polychromatic non-RGB data. Surprisingly, a minority of publications (n=39) made use of consumer-grade equipment. Lighting conditions were primarily uncontrolled or ambient. While some works focused on specialized aspects such as the removal of vital sign information from video streams to protect privacy or the influence of video compression, most algorithmic developments were related to three areas: region of interest selection, tracking, or extraction of a one-dimensional signal. Seven papers used deep learning techniques, 17 papers used other machine learning approaches, and 92 made no explicit use of machine learning.

CONCLUSION

Although some general trends and frequent shortcomings are obvious, the spectrum of publications related to camera-based vital sign estimation is broad. While many creative solutions and unique approaches exist, the lack of standardization hinders comparability of these techniques and of their performance. We believe that sharing algorithms and/ or datasets will alleviate this and would allow the application of newer techniques such as deep learning.

Cardiovascular assessment by imaging photoplethysmography - a review

Over the last few years, the contactless acquisition of cardiovascular parameters using cameras has gained immense attention. The technique provides an optical means to acquire cardiovascular information in a very convenient way. This review provides an overview on the technique's background and current realizations. Besides giving detailed information on the most widespread application of the technique, namely the contactless acquisition of heart rate, we outline further concepts and we critically discuss the current state.

Frame Registration for Motion Compensation in Imaging Photoplethysmography

Imaging photoplethysmography (iPPG) is an emerging technology used to assess microcirculation and cardiovascular signs by collecting backscattered light from illuminated tissue using optical imaging sensors. An engineering approach is used to evaluate whether a silicone cast of a human palm might be effectively utilized to predict the results of image registration schemes for motion compensation prior to their application on live human tissue. This allows us to establish a performance baseline for each of the algorithms and to isolate performance and noise fluctuations due to the induced motion from the temporally changing physiological signs. A multi-stage evaluation model is developed to qualitatively assess the influence of the region of interest (ROI), system resolution and distance, reference frame selection, and signal normalization on extracted iPPG waveforms from live tissue. We conclude that the application of image registration is able to deliver up to 75% signal-to-noise (SNR) improvement (4.75 to 8.34) over an uncompensated iPPG signal by employing an intensity-based algorithm with a moving reference frame.

Alterations of blood pulsations parameters in carotid basin due to body position change

The velocity of the pulse wave (PWV) propagating through the vascular tree is an essential parameter for diagnostic the state of the cardiovascular system especially when it is measured in the pool of carotid arteries. In this research, we showed for the first time that the time of the blood-pressure-wave propagation from the heart to the face is a function of the body position. Significant asymmetry and asynchronicity of blood pulsations in the facial area were found in a recumbent position. Parameters of blood pulsations were measured by an advanced camera-based photoplethysmography system in 73 apparently healthy subjects. Most likely, observed changes of the blood-pulsation parameters are caused by variations of the arterial blood pressure due to hydrostatic pressure changes, and secondary reaction of blood vessels in response to these variations. Demonstrated feasibility of PWV measurements in the pool of carotid arteries provides considerable advantages over other technologies. Moreover, possibilities of the method to estimate physiological regulation of the peripheral blood flow (particularly, as a response to the gravitational changes) have been demonstrated. The proposed concept allows development of non-invasive medical equipment capable of solving a wide range of scientific and practical problems related to vascular physiology.

Comparison of speckleplethysmographic (SPG) and photoplethysmographic (PPG) imaging by Monte Carlo simulations and in vivo measurements

Noncontact photoplethysmography (PPG) is limited by a poor signal-to-noise ratio (SNR). A solution to this limitation is the use of alternate sources of optical contrast to generate a complementary pulsatile waveform. One such source is laser speckle contrast, which is modulated in biological tissues by the flow rate of red blood cells. Averaging a region of interest from a speckle contrast image over time allows for the calculation of a speckleplethysmogram (SPG). Similar to PPG, SPG enables monitoring of heart rate and respiratory rate. A gap in the knowledge base exists as to the precise spatiotemporal relationship between PPG and SPG signals. We have developed an eight-layer tissue model to simulate both PPG and SPG signals in a reflectance geometry via Monte Carlo methods. We modeled PPG by compression of the upper and lower blood nets due to expansion of the larger arterial layer below. The in silico PPG peak-to-peak amplitude percent was greater at 532 nm than at 860 nm (5.6% vs. 3.0%, respectively), which matches trends from the literature. We modeled SPG by changing flow speeds of red blood cells in both the capillaries and arterioles over the cardiac cycle. The in silico SPG peak-to-peak amplitude percent was 24% at 532 nm and 40% at 860 nm. In silico results are similar to in vivo results measured with a two-camera set up for simultaneous imaging of PPG and SPG. Both in silico and in vivo data suggest SPG has a much larger SNR than PPG, which may prove beneficial for noncontact, wide-field optical monitoring of cardiovascular health.

Using imaging photoplethysmography for heart rate estimation in non-human primates

For humans and for non-human primates heart rate is a reliable indicator of an individual's current physiological state, with applications ranging from health checks to experimental studies of cognitive and emotional state. In humans, changes in the optical properties of the skin tissue correlated with cardiac cycles (imaging photoplethysmogram, iPPG) allow non-contact estimation of heart rate by its proxy, pulse rate. Yet, there is no established simple and non-invasive technique for pulse rate measurements in awake and behaving animals. Using iPPG, we here demonstrate that pulse rate in rhesus monkeys can be accurately estimated from facial videos. We computed iPPGs from eight color facial videos of four awake head-stabilized rhesus monkeys. Pulse rate estimated from iPPGs was in good agreement with reference data from a contact pulse-oximeter: the error of pulse rate estimation was below 5% of the individual average pulse rate in 83% of the epochs; the error was below 10% for 98% of the epochs. We conclude that iPPG allows non-invasive and non-contact estimation of pulse rate in non-human primates, which is useful for physiological studies and can be used toward welfare-assessment of non-human primates in research.

A Model for Waveform Dissimilarities in Dual-Depth Reflectance-PPG

The pressure wave is attenuated as it travels through the vascular bed of tissue. Consequently, reflectance photoplethysmography (PPG) waveforms probed using dual-penetrating wavelengths, such as green (G) and red R; the deepest) are dissimilar. To unravel the dual-depth aspect of PPG, we modeled the wavelength-dependency of the shape of reflection-PPG signals in G (520-580 nm) and R (625-720nm). Skin compression perturbs the relative contributions of the dermal and subdermal blood volume variations sources (BVVs) to PPG and was used to verify our model. We acquired reflectance-PPG in G and R on the finger of nine subjects (ages, 26-32 yrs). Two parameters were used for describing dual-depth dissimilarities: the phase shift, $\phi $, between the first harmonics of the subdermal and dermal BVVs, and the observed phase shift (PS) between PPG signals in G and R. The average $\phi $ was 37.6, CI 95% [22.0, 53.2] degrees. At uncompressed skin, this corresponds to an average PS of 12.5, [7.8, 17.2] degrees. Our results suggest that phase parameters may enable microvascular characterization and diagnosis.

Lock-in technique for extraction of pulse rates and associated confidence levels from video

We investigate the practical applicability of video photoplethysmography (VPPG) to extract heart rates of subjects using noncontact color video recordings of human faces collected under typical indoor laboratory conditions using commercial video cameras. Videos were processed following three previously described simple VPPG algorithms to produce a time-varying plethysmographic signal. These time signals were then analyzed using, to the best of our knowledge, a novel, lock-in algorithm that was developed to extract the pulsatile frequency component. A protocol to associate confidence estimates for the extracted heart rates for each video stream is presented. Results indicate that the difference between heart rates extracted using the lock-in technique and gold-standard measurements, for videos with high-confidence metrics, was less than 4 beats per minute. Constraints on video acquisition and processing, including natural subject motion and the total duration of video recorded required for evaluating these confidence metrics, are discussed.

New insights into the origin of remote PPG signals in visible light and infrared

Remote photoplethysmography (PPG) is an optical measurement technique with established applications in vital signs monitoring. Recently, the consensual understanding of blood volume variations (BVVs) as the origin of PPG signals was challenged, raising validity concerns about the remote SpO₂ methodology. Recognizing the imperative for new opto-physiological evidence, this investigation supports the volumetric hypothesis with living skin experiments and Monte Carlo simulations of remote PPG-amplitude in visible light (VIS) and infrared (IR). Multilayered models of the skin were developed to simulate the separate contributions from skin layers containing pulsatile arterioles to the PPG signal in the 450–1000 nm range. The simulated spectra were qualitatively compared with observations of the resting and compressed finger pad, and complemented with videocapillaroscopy. Our results indicate that remote PPG systems indeed probe arterial blood. Green wavelengths probe dermal arterioles while red-IR wavelengths also reach subcutaneous BVVs. Owing to stable penetration depths, the red-IR diagnostic window promotes the invariance of SpO₂ measurements to skin non-homogeneities.

Smart Garment Fabrics to Enable Non-Contact Opto-Physiological Monitoring

Imaging photoplethysmography (iPPG) is an emerging technology used to assess microcirculation and cardiovascular signs by collecting backscattered light from illuminated tissue using optical imaging sensors. The aim of this study was to study how effective smart garment fabrics could be capturing physiological signs in a non-contact mode. The present work demonstrates a feasible approach of, instead of using conventional high-power illumination sources, integrating a grid of surface-mounted light emitting diodes (LEDs) into cotton fabric to spotlight the region of interest (ROI). The green and the red LEDs (525 and 660 nm) placed on a small cotton substrate were used to locally illuminate palm skin in a dual-wavelength iPPG setup, where the backscattered light is transmitted to a remote image sensor through the garment fabric. The results show that the illuminations from both wavelength LEDs can be used to extract heart rate (HR) reaching an accuracy of 90% compared to a contact PPG probe. Stretching the fabric over the skin surface alters the morphology of iPPG signals, demonstrating a significantly higher pulsatile amplitude in both channels of green and red illuminations. The skin compression by the fabric could be potentially utilised to enhance the penetration of illumination into cutaneous microvascular beds. The outcome could lead a new avenue of non-contact opto-physiological monitoring and assessment with functional garment fabrics.

Camera-based photoplethysmography in an intraoperative setting

Background

Camera-based photoplethysmography (cbPPG) is a measurement technique which enables remote vital sign monitoring by using cameras. To obtain valid plethysmograms, proper regions of interest (ROIs) have to be selected in the video data. Most automated selection methods rely on specific spatial or temporal features limiting a broader application. In this work, we present a new method which overcomes those drawbacks and, therefore, allows cbPPG to be applied in an intraoperative environment.

Methods

We recorded 41 patients during surgery using an RGB and a near-infrared (NIR) camera. A Bayesian skin classifier was employed to detect suitable regions, and a level set segmentation approach to define and track ROIs based on spatial homogeneity.

Results

The results show stable and homogeneously illuminated ROIs. We further evaluated their quality with regards to extracted cbPPG signals. The green channel provided the best results where heart rates could be correctly estimated in 95.6% of cases. The NIR channel yielded the highest contribution in compensating false estimations.

Conclusions

The proposed method proved that cbPPG is applicable in intraoperative environments. It can be easily transferred to other settings regardless of which body site is considered.

Electronic supplementary material

The online version of this article (10.1186/s12938-018-0467-7) contains supplementary material, which is available to authorized users.

The value of polarization in camera-based photoplethysmography

Camera-based photoplethysmography (cbPPG) is a novel measuring technique that permits the remote acquisition of cardiovascular signals using video cameras. Research still lacks in fundamental studies to reach a deeper technical and physiological understanding. This work analyzes the employment of polarization filtration to (i) assess the gain for the signal quality and (ii) draw conclusions about the cbPPG signal's origin. We evaluated various forehead regions of 18 recordings with different color and filter settings. Our results prove that for an optimal illumination, the perpendicular filter setting provides a significant benefit. The outcome supports the theory that signals arise from blood volume changes. For lateral illumination, ballistocardiographic effects dominate the signal as polarization's impact vanishes.

Living-Skin Classification via Remote-PPG

Detecting living-skin tissue in a video on the basis of induced color changes due to blood pulsation is emerging for automatic region of interest localization in remote photoplethysmography (rPPG). However, the state-of-the-art method performing unsupervised living-skin detection in a video is rather time consuming, which is mainly due to the high complexity of its unsupervised online learning for pulse/noise separation. In this paper, we address this issue by proposing a fast living-skin classification method. Our basic idea is to transform the time-variant rPPG-signals into signal shape descriptors called "multiresolution iterative spectrum," where pulse and noise have different patterns enabling accurate binary classification. The proposed technique is a proof-of-concept that has only been validated in lab conditions but not in real clinical conditions. The benchmark, including synthetic and realistic (nonclinical) experiments, shows that it achieves a high detection accuracy better than the state-of-the-art method, and a high detection speed at hundreds of frames per second in MATLAB, enabling real-time living-skin detection.

New principle for measuring arterial blood oxygenation, enabling motion-robust remote monitoring

Finger-oximeters are ubiquitously used for patient monitoring in hospitals worldwide. Recently, remote measurement of arterial blood oxygenation (SpO₂) with a camera has been demonstrated. Both contact and remote measurements, however, require the subject to remain static for accurate SpO₂ values. This is due to the use of the common ratio-of-ratios measurement principle that measures the relative pulsatility at different wavelengths. Since the amplitudes are small, they are easily corrupted by motion-induced variations. We introduce a new principle that allows accurate remote measurements even during significant subject motion. We demonstrate the main advantage of the principle, i.e. that the optimal signature remains the same even when the SNR of the PPG signal drops significantly due to motion or limited measurement area. The evaluation uses recordings with breath-holding events, which induce hypoxemia in healthy moving subjects. The events lead to clinically relevant SpO₂ levels in the range 80–100%. The new principle is shown to greatly outperform current remote ratio-of-ratios based methods. The mean-absolute SpO₂-error (MAE) is about 2 percentage-points during head movements, where the benchmark method shows a MAE of 24 percentage-points. Consequently, we claim ours to be the first method to reliably measure SpO₂ remotely during significant subject motion.

Skin inhomogeneity as a source of error in remote PPG-imaging

Remote photoplethysmography (rPPG) imaging is an optical technique to remotely assess the local cutaneous microcirculation. Despite its potential for enabling health-related applications, the current understanding of the outcome images/maps remains incomplete. In this paper, we present a model and supporting experiments confirming the contribution of skin inhomogeneity to the morphology of PPG waveforms. Since rPPG imagers rely on the complex inner-product operator and may combine multiple wavelengths, the derived phase measurements reflect morphological heterogeneity of PPG signals to a larger extent than propagation-related phase differences. The influence of light penetration depth on PPG was observed and modeled on the green and red wavelengths at the hand region. We further show how our work contributes to understanding reproducibility issues in recent papers on pulse wave velocity (PWV) estimation.