Non-contact detection of oxygen saturation based on visible light imaging device using ambient light

A Study of Data Processing Methods for Non-Contact Multispectral Method of Blood Oxygen Saturation

Regular monitoring of blood oxygenation is important for disease prevention and treatment. Image photoplethysmography (IPPG) technology is a non-contact physiological parameter detection technology, which has been widely used in blood oxygenation detection. However, traditional imaging devices still have issues such as low detection accuracy, narrower receiving spectral range. In this paper, we proposed two improved detection methods based on the dual-wavelength measurement principle, that is, dual-band IPPG signal ratio method and dual-band IPPG signal AC/DC method. To verify the effectiveness of the two methods, we used different heartbeat period IPPG signals as sample data sets, and combined PLS and RF algorithms for model training, thus obtaining the best data processing method. The experimental results showed that the dual-band IPPG signal AC/DC method can effectively reduce the model training time. This method meets the strong demand for non-contact blood oxygen measurement and provides a new measurement idea.

CMOS-Compatible High-Performance Silicon Nanowire Array Natural Light Electronic Detection System

In this article, we propose a novel natural light detector based on high-performance silicon nanowire (SiNW) arrays. We achieved a highly controllable and low-cost fabrication of SiNW natural light detectors by using only a conventional micromachined CMOS process. The high activity of SiNWs leads to the poor long-term stability of the SiNW device, and for this reason, we have designed a fully wrapped structure for SiNWs. SiNWs are wrapped in transparent silicon nitride and silicon oxide films, which greatly improves the long-term stability of the detector; at the same time, this structure protects the SiNWs from breakage. In addition, the SiNW arrays are regularly distributed on the top of the detector, which can quickly respond to natural light. The response time of the detector is about 0.015 s. Under the illumination of 1 W·m-2 light intensity, multiple SiNWs were detected together. The signal strength of the detector reached 1.82 μA, the signal-to-noise ratio was 47.6 dB, and the power consumption was only 0.91 μW. The high-intensity and highly reliable initial signal reduces the cost and complexity of the backend signal processing circuit. A low-cost and high-performance STM32 microcontroller can realize the signal processing task. Therefore, we built a high-performance SiNW natural optoelectronic detection system based on an STM32 microcontroller, which achieved the real-time detection of natural light intensity, with an accuracy of ±0.1 W·m-2. These excellent test performances indicate that the SiNW array natural light detector in this article meey the requirements of practicality and has broad potential for application.

Investigation of a Camera-Based Contactless Pulse Oximeter with Time-Division Multiplex Illumination Applied on Piglets for Neonatological Applications

(1) Objective: This study aims to lay a foundation for noncontact intensive care monitoring of premature babies. (2) Methods: Arterial oxygen saturation and heart rate were measured using a monochrome camera and time-division multiplex controlled lighting at three different wavelengths (660 nm, 810 nm and 940 nm) on a piglet model. (3) Results: Using this camera system and our newly designed algorithm for further analysis, the detection of a heartbeat and the calculation of oxygen saturation were evaluated. In motionless individuals, heartbeat and respiration were separated clearly during light breathing and with only minor intervention. In this case, the mean difference between noncontact and contact saturation measurements was 0.7% (RMSE = 3.8%, MAE = 2.93%). (4) Conclusions: The new sensor was proven effective under ideal animal experimental conditions. The results allow a systematic improvement for the further development of contactless vital sign monitoring systems. The results presented here are a major step towards the development of an incubator with noncontact sensor systems for use in the neonatal intensive care unit.

Innovative approaches in imaging photoplethysmography for remote blood oxygen monitoring

Peripheral Capillary Oxygen Saturation (SpO2) has received increasing attention during the COVID-19 pandemic. Clinical investigations have demonstrated that individuals afflicted with COVID-19 exhibit notably reduced levels of SpO2 before the deterioration of their health status. To cost-effectively enable individuals to monitor their SpO2, this paper proposes a novel neural network model named "ITSCAN" based on Temporal Shift Module. Benefiting from the widespread use of smartphones, this model can assess an individual's SpO2 in real time, utilizing standard facial video footage, with a temporal granularity of seconds. The model is interweaved by two distinct branches: the motion branch, responsible for extracting spatiotemporal data features and the appearance branch, focusing on the correlation between feature channels and the location information of feature map using coordinate attention mechanisms. Accordingly, the SpO2 estimator generates the corresponding SpO2 value. This paper summarizes for the first time 5 loss functions commonly used in the SpO2 estimation model. Subsequently, a novel loss function has been contributed through the examination of various combinations and careful selection of hyperparameters. Comprehensive ablation experiments analyze the independent impact of each module on the overall model performance. Finally, the experimental results based on the public dataset (VIPL-HR) show that our model has obvious advantages in MAE (1.10%) and RMSE (1.19%) compared with related work, which implies more accuracy of the proposed method to contribute to public health.

Deep learning and remote photoplethysmography powered advancements in contactless physiological measurement

In recent decades, there has been ongoing development in the application of computer vision (CV) in the medical field. As conventional contact-based physiological measurement techniques often restrict a patient's mobility in the clinical environment, the ability to achieve continuous, comfortable and convenient monitoring is thus a topic of interest to researchers. One type of CV application is remote imaging photoplethysmography (rPPG), which can predict vital signs using a video or image. While contactless physiological measurement techniques have an excellent application prospect, the lack of uniformity or standardization of contactless vital monitoring methods limits their application in remote healthcare/telehealth settings. Several methods have been developed to improve this limitation and solve the heterogeneity of video signals caused by movement, lighting, and equipment. The fundamental algorithms include traditional algorithms with optimization and developing deep learning (DL) algorithms. This article aims to provide an in-depth review of current Artificial Intelligence (AI) methods using CV and DL in contactless physiological measurement and a comprehensive summary of the latest development of contactless measurement techniques for skin perfusion, respiratory rate, blood oxygen saturation, heart rate, heart rate variability, and blood pressure.

Contactless Blood Oxygen Saturation Estimation from Facial Videos Using Deep Learning

Blood oxygen saturation (SpO2) is an essential physiological parameter for evaluating a person's health. While conventional SpO2 measurement devices like pulse oximeters require skin contact, advanced computer vision technology can enable remote SpO2 monitoring through a regular camera without skin contact. In this paper, we propose novel deep learning models to measure SpO2 remotely from facial videos and evaluate them using a public benchmark database, VIPL-HR. We utilize a spatial-temporal representation to encode SpO2 information recorded by conventional RGB cameras and directly pass it into selected convolutional neural networks to predict SpO2. The best deep learning model achieves 1.274% in mean absolute error and 1.71% in root mean squared error, which exceed the international standard of 4% for an approved pulse oximeter. Our results significantly outperform the conventional analytical Ratio of Ratios model for contactless SpO2 measurement. Results of sensitivity analyses of the influence of spatial-temporal representation color spaces, subject scenarios, acquisition devices, and SpO2 ranges on the model performance are reported with explainability analyses to provide more insights for this emerging research field.

CL-SPO2Net: Contrastive Learning Spatiotemporal Attention Network for Non-Contact Video-Based SpO2 Estimation

Video-based peripheral oxygen saturation (SpO2) estimation, utilizing solely RGB cameras, offers a non-contact approach to measuring blood oxygen levels. Previous studies set a stable and unchanging environment as the premise for non-contact blood oxygen estimation. Additionally, they utilized a small amount of labeled data for system training and learning. However, it is challenging to train optimal model parameters with a small dataset. The accuracy of blood oxygen detection is easily affected by ambient light and subject movement. To address these issues, this paper proposes a contrastive learning spatiotemporal attention network (CL-SPO2Net), an innovative semi-supervised network for video-based SpO2 estimation. Spatiotemporal similarities in remote photoplethysmography (rPPG) signals were found in video segments containing facial or hand regions. Subsequently, integrating deep neural networks with machine learning expertise enabled the estimation of SpO2. The method had good feasibility in the case of small-scale labeled datasets, with the mean absolute error between the camera and the reference pulse oximeter of 0.85% in the stable environment, 1.13% with lighting fluctuations, and 1.20% in the facial rotation situation.

Notch RGB-camera based SpO2 estimation: a clinical trial in a neonatal intensive care unit

Regular and narrow-band RGB cameras are recently explored for contactless SpO2 monitoring. Regular RGB cameras with cross-band overlap provide a high signal-to-noise-ratio (SNR) in measuring the photoplethysmographic signals but possess high dependency on the spectra of incident light, whereas narrow-band RGB cameras have better spectral independence but lower SNR especially in dim lighting conditions, such as in the neonatal intensive care unit (NICU). This paper proposes a notch RGB camera based SpO2 measurement approach that uses an optical notch filter to attenuate the wavelengths of 580-605 nm of a regular RGB camera to improve the spectral independence while maintaining high SNR in signal measurement. The proposed setup was validated in the lab condition (e.g. dark chamber) against the existing solutions for visible-light based camera-SpO2 measurement and further verified in the NICU on preterm infants. The clinical trial conducted in the NICU with 22 preterm infants shows that the notch RGB camera can achieve a mean absolute error (MAE) less than 4% for SpO2 measurement. This is the first showcase of continuous monitoring of absolute camera-SpO2 values in the NICU.

Remote Blood Oxygen Estimation From Videos Using Neural Networks

Peripheral blood oxygen saturation (SpO 2) is an essential indicator of respiratory functionality and received increasing attention during the COVID-19 pandemic. Clinical findings show that COVID-19 patients can have significantly low SpO 2 before any obvious symptoms. Measuring an individual's SpO 2 without having to come into contact with the person can lower the risk of cross contamination and blood circulation problems. The prevalence of smartphones has motivated researchers to investigate methods for monitoring SpO 2 using smartphone cameras. Most prior schemes involving smartphones are contact-based: They require using a fingertip to cover the phone's camera and the nearby light source to capture reemitted light from the illuminated tissue. In this paper, we propose the first convolutional neural network based noncontact SpO 2 estimation scheme using smartphone cameras. The scheme analyzes the videos of an individual's hand for physiological sensing, which is convenient and comfortable for users and can protect their privacy and allow for keeping face masks on. We design explainable neural network architectures inspired by the optophysiological models for SpO 2 measurement and demonstrate the explainability by visualizing the weights for channel combination. Our proposed models outperform the state-of-the-art model that is designed for contact-based SpO 2 measurement, showing the potential of the proposed method to contribute to public health. We also analyze the impact of skin type and the side of a hand on SpO 2 estimation performance.

Impact of Different Skin Penetration Depths of Red and Green Wavelengths on Camera-based SpO2 Measurement

Remote camera-based estimation of blood oxygen saturation (SpO2) using visible lights has been studied recently, typically for red (660 nm) and green (550 nm) wavelengths. This paper investigates the impact of different skin penetration depths of red and green wavelengths on the SpO2 estimation based on mathematical modeling and experiments, where the SpO2-calibritability between two illumination setups, narrow-band red/green and narrow-band red/infrared, are statistically compared using the "ratio-of-ratios" method. The results show that the performance of the setup using red/green is less consistent among 17 volunteers than the setup using red/infrared, and larger SpO2 disparity between different skin regions (by SpO2 imaging) have been found for individuals in the red/green wavelengths setup. The use of visible light (red and green) may impose a risk of SpO2 calibration due to the different skin penetration depths of these two wavelengths.

Prediction and prevention system for Severe Acute Respiratory Syndrome CoronaVirus 2 infection by preempting the onset of a cough

The severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection is fast becoming one of the most significant infections worldwide. Of all the causes of SARS-CoV-2 infection, airborne-droplet infection via coughing is the most common. Therefore, if predicting the onset of a cough and preventing infection were possible, it would have a globally positive impact. Here, we describe a new prediction and prevention system for SARS-CoV-2 infection. Usually, air is inhaled prior to coughing, and the cough, which contains droplets of the virus, then occurs during acute exhalation. Therefore, if we can predict the onset of a cough, we can prevent the spread of SARS-CoV-2. At Tohoku University, a diagnosis system for evaluating swallowing motions and peripheral circulation has already been developed, and our prediction system can be integrated into this system. Using three-dimensional human body imaging, we developed a prediction system for preempting the onset of a cough. If we can predict the onset a cough, we can prevent the spread of SARS-CoV-2 infection, by decreasing the shower of virally active airborne droplets. Here, we describe the newly developed prediction and prevention system for SARS-CoV-2 infection that preempts the onset of a cough.Clinical Relevance- If predicting the onset of a cough and preventing infection were possible, it would have a globally positive impact. Here, we describe the newly developed prediction and prevention system for SARS-CoV-2 infection.

Current methods for contactless optical patient diagnosis: a systematic review

Many countries around the world face a shortage of medical personnel, leading to work overload or even burnout. This calls for political and scientific solutions to relieve the medical personnel. The measurement of vital signs in hospitals is still predominately carried out manually with traditional contact-based methods, taking over a substantial share of the medical personnel's workload. The introduction of contactless methods for vital sign monitoring (e.g., with a camera) has great potential to relieve the medical personnel. This systematic review's objective is to analyze the state of the art in the field of contactless optical patient diagnosis. This review distinguishes itself from already existing reviews by considering studies that do not only propose the contactless measurement of vital signs but also include an automatic diagnosis of the patient's condition. This means that the included studies incorporate the physician's reasoning and evaluation of vital signs into their algorithms, allowing an automated patient diagnosis. The literature screening of two independent reviewers resulted in a total of five eligible studies. The highest number of studies (three) introduce methods for the risk assessment of infectious diseases, one study introduces a method for the risk assessment of cardiovascular diseases, and one study introduces a method for the diagnosis of obstructive sleep apnea. Overall, high heterogeneity in relevant study parameters is reported among the included studies. The low number of included studies indicates a large research gap and emphasizes the demand for further research on this emerging topic.

Deep Learning-Based Non-Contact IPPG Signal Blood Pressure Measurement Research

In this paper, a multi-stage deep learning blood pressure prediction model based on imaging photoplethysmography (IPPG) signals is proposed to achieve accurate and convenient monitoring of human blood pressure. A camera-based non-contact human IPPG signal acquisition system is designed. The system can perform experimental acquisition under ambient light, effectively reducing the cost of non-contact pulse wave signal acquisition while simplifying the operation process. The first open-source dataset IPPG-BP for IPPG signal and blood pressure data is constructed by this system, and a multi-stage blood pressure estimation model combining a convolutional neural network and bidirectional gated recurrent neural network is designed. The results of the model conform to both BHS and AAMI international standards. Compared with other blood pressure estimation methods, the multi-stage model automatically extracts features through a deep learning network and combines different morphological features of diastolic and systolic waveforms, which reduces the workload while improving accuracy.

Using Contactless Facial Image Recognition Technology to Detect Blood Oxygen Saturation

Since the outbreak of COVID-19, as of January 2023, there have been over 670 million cases and more than 6.8 million deaths worldwide. Infections can cause inflammation in the lungs and decrease blood oxygen levels, which can lead to breathing difficulties and endanger life. As the situation continues to escalate, non-contact machines are used to assist patients at home to monitor their blood oxygen levels without encountering others. This paper uses a general network camera to capture the forehead area of a person's face, using the RPPG (remote photoplethysmography) principle. Then, image signal processing of red and blue light waves is carried out. By utilizing the principle of light reflection, the standard deviation and mean are calculated, and the blood oxygen saturation is computed. Finally, the effect of illuminance on the experimental values is discussed. The experimental results of this paper were compared with a blood oxygen meter certified by the Ministry of Health and Welfare in Taiwan, and the experimental results had only a maximum error of 2%, which is better than the 3% to 5% error rates in other studies The measurement time was only 30 s, which is better than the one minute reported using similar equipment in other studies. Therefore, this paper not only saves equipment expenses but also provides convenience and safety for those who need to monitor their blood oxygen levels at home. Future applications can combine the SpO2 detection software with camera-equipped devices such as smartphones and laptops. The public can detect SpO2 on their own mobile devices, providing a convenient and effective tool for personal health management.

Non-contact oxygen saturation monitoring for wound healing process using dual-wavelength simultaneous acquisition imaging system

Here we report the fabrication of a noncontact pulse oximeter system based on a dual-wavelength imaging system and its oxygen saturation monitoring performance during wound healing. The dual-wavelength imaging system consists of 660 nm and 940 nm light-emitting diodes and a multi-spectral camera that simultaneously accepts visible and near-infrared images. Using the proposed system, images were acquired at 30 fps at both wavelengths, and photoplethysmography signals were extracted by specifying a specific region in the images. We removed the signals caused by small movements and smoothed them using the discrete wavelet transform and moving average filter. To confirm the feasibility of the proposed noncontact oxygen saturation system, a wound model was created using a hairless mouse and oxygen saturation was measured during wound healing. The measured values were compared and analyzed using a reflective animal pulse oximeter. Through a comparative analysis of these two devices, the error of the proposed system was evaluated and the possibility of its clinical application and wound healing monitoring through oxygen saturation measurement confirmed.

LSTM-based real-time signal quality assessment for blood volume pulse analysis

Remote photoplethysmogram (rPPG) is a low-cost method to extract blood volume pulse (BVP). Some crucial vital signs, such as heart rate (HR) and respiratory rate (RR) etc. can be achieved from BVP for clinical medicine and healthcare application. As compared to the conventional PPG methods, rPPG is more promising because of its non-contacted measurement. However, both BVP detection methods, especially rPPG, are susceptible to motion and illumination artifacts, which lead to inaccurate estimation of vital signs. Signal quality assessment (SQA) is a method to measure the quality of BVP signals and ensure the credibility of estimated physiological parameters. But the existing SQA methods are not suitable for real-time processing. In this paper, we proposed an end-to-end BVP signal quality evaluation method based on a long short-term memory network (LSTM-SQA). Two LSTM-SQA models were trained using the BVP signals obtained with PPG and rPPG techniques so that the quality of BVP signals derived from these two methods can be evaluated, respectively. As there is no publicly available rPPG dataset with quality annotations, we designed a training sample generation method with blind source separation, by which two kinds of training datasets respective to PPG and rPPG were built. Each dataset consists of 38400 high and low-quality BVP segments. The achieved models were verified on three public datasets (IIP-HCI dataset, UBFC-Phys dataset, and LGI-PPGI dataset). The experimental results show that the proposed LSTM-SQA models can effectively predict the quality of the BVP signal in real-time.

Contactless blood oxygen estimation from face videos: A multi-model fusion method based on deep learning

Blood Oxygen ( SpO 2 ), a key indicator of respiratory function, has received increasing attention during the COVID-19 pandemic. Clinical results show that patients with COVID-19 likely have distinct lower SpO 2 before the onset of significant symptoms. Aiming at the shortcomings of current methods for monitoring SpO 2 by face videos, this paper proposes a novel multi-model fusion method based on deep learning for SpO 2 estimation. The method includes the feature extraction network named Residuals and Coordinate Attention (RCA) and the multi-model fusion SpO 2 estimation module. The RCA network uses the residual block cascade and coordinate attention mechanism to focus on the correlation between feature channels and the location information of feature space. The multi-model fusion module includes the Color Channel Model (CCM) and the Network-Based Model(NBM). To fully use the color feature information in face videos, an image generator is constructed in the CCM to calculate SpO 2 by reconstructing the red and blue channel signals. Besides, to reduce the disturbance of other physiological signals, a novel two-part loss function is designed in the NBM. Given the complementarity of the features and models that CCM and NBM focus on, a Multi-Model Fusion Model(MMFM) is constructed. The experimental results on the PURE and VIPL-HR datasets show that three models meet the clinical requirement(the mean absolute error ⩽ 2%) and demonstrate that the multi-model fusion can fully exploit the SpO 2 features of face videos and improve the SpO 2 estimation performance. Our research achievements will facilitate applications in remote medicine and home health.

Contactless SpO2 with an RGB camera: experimental proof of calibrated SpO2

Camera-based blood oxygen saturation (SpO2) monitoring allows reliable measurements without touching the skin and is therefore very attractive when there is a risk of cross-infection, in case of fragile skin, and/or to improve the clinical workflow. Despite promising results, productization of the technology is hampered by the unavailability of adequate hardware, especially a camera, which can capture the optimal wavelengths for SpO2 measurements in the red near-infrared region. A regular color (RGB) camera is attractive because of its availability, but also poses several risks and challenges which affect the accuracy of the measurement. To mitigate the most important risks, we propose to add low-cost commercial off-the-shelf (COTS) components to the setup. We executed two studies with this setup: one at a hypoxia lab with SpO2 values in the range 70 - 100% with the purpose to determine the calibration model, and the other study on volunteers to investigate the accuracy for different spot-check scenarios. The proposed processing pipeline includes face tracking and a robust method to estimate the ratio of relative amplitudes of the photoplethysmographic waveforms. Results show that the error is smaller than 4 percent points for realistic screening scenarios where the subject is seated, either with or without head support and/or ambient light.

A non-contact oxygen saturation detection method based on dynamic spectrum

Blood oxygen saturation (SpO2) is an important monitoring indicator for many respiratory diseases. Non-contact oximetry offers outstanding advantages in both coronavirus pandemic monitoring and sleep monitoring, but at the same time poses both challenges regarding technology and environment. Therefore, we propose a method for non-contact SpO2 measurement based on the principle of DS (dynamic spectrum) in this paper. A multispectral camera with 24 wavelengths (range in 660 nm-950 nm) is used to capture video of the people's cheek region, and then the two-dimensional images are converted into a one-dimensional temporal PPG signal. After pre-processing the PPG signal, the 24 wavelengths DS values are extracted. The optimal wavelength combination is obtained by wavelength screening using the one-by-one elimination method, and a PLS (partial least squares) model is established using the SpO2 values measured simultaneously by pulse oximetry as the modeled true values. The facial videos of eight healthy subjects were collected, and a total of 140 valid samples were obtained. By analyzing the modeling results, the regression coefficient (R) and root mean square error (RMSE) of the modeled set were 0.6366 and 0.9906, respectively. This method can significantly respond to the variation of SpO2, and the prediction results are approaching to the prediction accuracy (±2%) of most pulse oximeters in the market. Using DS theory in this method eliminates in principle the interference of static tissue, individual differences, and environment. It fully meets the strong demand for non-contact oximetry and provides a new measurement idea.

Photoplethysmographic Imaging of Hemodynamics and Two-Dimensional Oximetry

The review of recent papers devoted to actively developing methods of photoplethysmographic imaging (the PPGI) of blood volume pulsations in vessels and non-contact two-dimensional oximetry on the surface of a human body has been carried out. The physical fundamentals and technical aspects of the PPGI and oximetry have been considered. The manifold of the physiological parameters available for the analysis by the PPGI method has been shown. The prospects of the PPGI technology have been discussed. The possibilities of non-contact determination of blood oxygen saturation SpO2 (pulse saturation O2) have been described. The relevance of remote determination of the level of oxygenation in connection with the spread of a new coronavirus infection SARS-CoV-2 (COVID-19) has been emphasized. Most of the works under consideration cover the period 2010-2021.

Assessment of Intestinal Ischemia-Reperfusion Injury Using Diffuse Reflectance VIS-NIR Spectroscopy and Histology

A porcine model was used to investigate the feasibility of using VIS-NIR spectroscopy to differentiate between degrees of ischemia-reperfusion injury in the small intestine. Ten pigs were used in this study and four segments were created in the small intestine of each pig: (1) control, (2) full arterial and venous mesenteric occlusion for 8 h, (3) arterial and venous mesenteric occlusion for 2 h followed by reperfusion for 6 h, and (4) arterial and venous mesenteric occlusion for 4 h followed by reperfusion for 4 h. Two models were built using partial least square discriminant analysis. The first model was able to differentiate between the control, ischemic, and reperfused intestinal segments with an average accuracy of 99.2% with 10-fold cross-validation, and the second model was able to discriminate between the viable versus non-viable intestinal segments with an average accuracy of 96.0% using 10-fold cross-validation. Moreover, histopathology was used to investigate the borderline between viable and non-viable intestinal segments. The VIS-NIR spectroscopy method together with a PLS-DA model showed promising results and appears to be well-suited as a potentially real-time intraoperative method for assessing intestinal ischemia-reperfusion injury, due to its easy-to-use and non-invasive nature.

Noninvasive Non-Contact SpO2 Monitoring Using an Integrated Polarization-Sensing CMOS Imaging Sensor

BACKGROUND

In the diagnosis and primary health care of an individual, estimation of the pulse rate and blood oxygen saturation (SpO2) is critical. The pulse rate and SpO2 are determined by methods including photoplethysmography (iPPG), light spectroscopy, and pulse oximetry. These devices need to be compact, non-contact, and noninvasive for real-time health monitoring. Reflection-based iPPG is becoming popular as it allows non-contact estimation of the heart rate and SpO2. Most iPPG methods capture temporal data and form complex computations, and thus real-time measurements and spatial visualization are difficult.

METHOD

In this research work, reflective mode polarized imaging-based iPPG is proposed. For polarization imaging, a custom image sensor with wire grid polarizers on each pixel is designed. Each pixel has a wire grid of varying transmission axes, allowing phase detection of the incoming light. The phase information of the backscattered light from the fingertips of 12 healthy volunteers was recorded in both the resting as well as the excited states. These data were then processed using MATLAB 2021b software.

RESULTS

The phase information provides quantitative information on the reflection from the superficial and deep layers of skin. The ratio of deep to superficial layer backscattered phase information is shown to be directly correlated and linearly increasing with an increase in the SpO2 and heart rate.

CONCLUSIONS

The phase-based measurements help to monitor the changes in the resting and excited state heart rate and SpO2 in real time. Furthermore, the use of the ratio of phase information helps to make the measurements independent of the individual skin traits and thus increases the accuracy of the measurements. The proposed iPPG works in ambient light, relaxing the instrumentation requirement and helping the system to be compact and portable.

Availability and performance of face based non-contact methods for heart rate and oxygen saturation estimations: A systematic review

BACKGROUND

Consumer-level cameras have provided an advantage of designing cost-effective, non-contact physiological parameters estimation approaches which is not possible with gold standard estimation techniques. This encourages the development of non-contact estimation methods using camera technology. Therefore, this work aims to present a systematic review summarizing the currently existing face-based non-contact methods along with their performance.

METHODS

This review includes all heart rate (HR) and oxygen saturation (SpO2) studies published in journals and a few reputed conferences, which have compared the proposed estimation methods with one or more standard reference devices. The articles were collected from the following research databases: Institute of Electrical and Electronics Engineers (IEEE), PubMed, Web of Science (WoS), Science Direct, and Association of Computer Machinery (ACM) digital library. All database searches were completed on May 20, 2021. Each study was assessed using a finite set of identified factors for reporting bias.

RESULTS

Out of 332 identified studies, 32 studies were selected for the final review. Additionally, 18 studies were included by thoroughly checking these studies. 3 out of 50 (6%) studies were performed in clinical conditions, while the remaining studies were carried out on a healthy population. 42 out of 50 (84%) studies have estimated HR, while 5/50 (10%) studies have measured SpO₂ only. The remaining three studies have estimated both parameters. The majority of the studies have used 1-3 min videos for estimation. Among the estimation methods, Deep Learning and Independent component analysis (ICA) were used by 11/42 (26.19%) and 9/42 (21.42%) studies, respectively. According to the Bland-Altman analysis, only 8/45 (17.77%) HR studies achieved the clinically accepted error limits whereas, for SpO₂, 4/5 (80%) studies have matched the industry standards (±3%).

DISCUSSION

Deep Learning and ICA have been predominantly used for HR estimations. Among deep learning estimation methods, convolutional neural networks have been employed till date due to their good generalization ability. Most non-contact HR estimation methods need significant improvements to implement these methods in a clinical environment. Furthermore, these methods need to be tested on the subjects suffering from any related disease. SpO₂ estimation studies are challenging and need to be tested by conducting hypoxemic events. The authors would encourage reporting the detailed information about the study population, the use of longer videos, and appropriate performance metrics and testing under abnormal HR and SpO₂ ranges for future estimation studies.

Contactless Vital Signs Monitoring From Videos Recorded With Digital Cameras: An Overview

The measurement of physiological parameters is fundamental to assess the health status of an individual. The contactless monitoring of vital signs may provide benefits in various fields of application, from healthcare and clinical setting to occupational and sports scenarios. Recent research has been focused on the potentiality of camera-based systems working in the visible range (380-750 nm) for estimating vital signs by capturing subtle color changes or motions caused by physiological activities but invisible to human eyes. These quantities are typically extracted from videos framing some exposed body areas (e.g., face, torso, and hands) with adequate post-processing algorithms. In this review, we provided an overview of the physiological and technical aspects behind the estimation of vital signs like respiratory rate, heart rate, blood oxygen saturation, and blood pressure from digital images as well as the potential fields of application of these technologies. Per each vital sign, we provided the rationale for the measurement, a classification of the different techniques implemented for post-processing the original videos, and the main results obtained during various applications or in validation studies. The available evidence supports the premise of digital cameras as an unobtrusive and easy-to-use technology for physiological signs monitoring. Further research is needed to promote the advancements of the technology, allowing its application in a wide range of population and everyday life, fostering a biometrical holistic of the human body (BHOHB) approach.

Non-invasively accuracy enhanced blood glucose sensor using shallow dense neural networks with NIR monitoring and medical features

Non-invasive and accurate method for continuous blood glucose monitoring, the self-testing of blood glucose is in quest for better diagnosis, control and the management of diabetes mellitus (DM). Therefore, this study reports a multiple photonic band near-infrared (mbNIR) sensor augmented with personalized medical features (PMF) in Shallow Dense Neural Networks (SDNN) for the precise, inexpensive and pain free blood glucose determination. Datasets collected from 401 blood samples were randomized and trained with ten-fold validation. Additionally, a cohort of 234 individuals not included in the model training set were investigated to evaluate the performance of the model. The model achieved the accuracy of 97.8% along with 96.0% precision, 94.8% sensitivity and 98.7% specificity for DM classification based on a diagnosis threshold of 126 mg/dL for diabetes in fasting blood glucose. For non-invasive real-time blood glucose monitoring, the model exhibited ± 15% error with 95% confidence interval and the detection limit of 60–400 mg/dL, as validated with the standard hexokinase enzymatic method for glucose estimation. In conclusion, this proposed mbNIR based SDNN model with PMF is highly accurate and computationally cheaper compared to similar previous works using complex neural network. Some groups proposed using complicated mixed types of sensors to improve noninvasive glucose prediction accuracy; however, the accuracy gain over the complexity and costs of the systems harvested is still in questioned (Geng et al. in Sci Rep 7:12650, 2017). None of previous works report on accuracy enhancement of NIR/NN using PMF. Therefore, the proposed SDNN over PMF/mbNIR is an extremely promising candidate for the non-invasive real-time blood glucose monitoring with less complexity and pain-free.

Evaluating the robustness of a contact-less mHealth solution for personal and remote monitoring of blood oxygen saturation

MHealth technologies play a fundamental role in epidemiological situations such as the ongoing outbreak of COVID-19 because they allow people to self-monitor their health status (e.g. vital parameters) at any time and place, without necessarily having to physically go to a medical clinic. Among vital parameters, special care should be given to monitor blood oxygen saturation (SpO2), whose abnormal values are a warning sign for potential COVID-19 infection. SpO2 is commonly measured through the pulse oximeter that requires skin contact and hence could be a potential way of spreading contagious infections. To overcome this problem, we have recently developed a contact-less mHealth solution that can measure blood oxygen saturation without any contact device but simply processing short facial videos acquired by any common mobile device equipped with a camera. Facial video frames are processed in real-time to extract the remote photoplethysmographic signal useful to estimate the SpO2 value. Such a solution promises to be an easy-to-use tool for both personal and remote monitoring of SpO2. However, the use of mobile devices in daily situations holds some challenges in comparison to the controlled laboratory scenarios. One main issue is the frequent change of perspective viewpoint due to head movements, which makes it more difficult to identify the face and measure SpO2. The focus of this work is to assess the robustness of our mHealth solution to head movements. To this aim, we carry out a pilot study on the benchmark PURE dataset that takes into account different head movements during the measurement. Experimental results show that the SpO2 values obtained by our solution are not only reliable, since they are comparable with those obtained with a pulse oximeter, but are also insensitive to head motion, thus allowing a natural interaction with the mobile acquisition device.

Deep Learning Methods for Remote Heart Rate Measurement: A Review and Future Research Agenda

Heart rate (HR) is one of the essential vital signs used to indicate the physiological health of the human body. While traditional HR monitors usually require contact with skin, remote photoplethysmography (rPPG) enables contactless HR monitoring by capturing subtle light changes of skin through a video camera. Given the vast potential of this technology in the future of digital healthcare, remote monitoring of physiological signals has gained significant traction in the research community. In recent years, the success of deep learning (DL) methods for image and video analysis has inspired researchers to apply such techniques to various parts of the remote physiological signal extraction pipeline. In this paper, we discuss several recent advances of DL-based methods specifically for remote HR measurement, categorizing them based on model architecture and application. We further detail relevant real-world applications of remote physiological monitoring and summarize various common resources used to accelerate related research progress. Lastly, we analyze the implications of research findings and discuss research gaps to guide future explorations.

Non-Contact Oxygen Saturation Measurement Using YCgCr Color Space with an RGB Camera

Oxygen saturation (SPO₂) is an important indicator of health, and is usually measured by placing a pulse oximeter in contact with a finger or earlobe. However, this method has a problem in that the skin and the sensor must be in contact, and an additional light source is required. To solve these problems, we propose a non-contact oxygen saturation measurement technique that uses a single RGB camera in an ambient light environment. Utilizing the fact that oxygenated and deoxygenated hemoglobin have opposite absorption coefficients at green and red wavelengths, the color space of photoplethysmographic (PPG) signals recorded from the faces of study participants were converted to the YCgCr color space. Substituting the peaks and valleys extracted from the converted Cg and Cr PPG signals into the Beer–Lambert law yields the SPO₂ via a linear equation. When the non-contact SPO₂ measurement value was evaluated based on the reference SPO₂ measured with a pulse oximeter, the mean absolute error was 0.537, the root mean square error was 0.692, the Pearson correlation coefficient was 0.86, the cosine similarity was 0.99, and the intraclass correlation coefficient was 0.922. These results confirm the feasibility of non-contact SPO₂ measurements.

Analysis and improvement of non-contact SpO2 extraction using an RGB webcam

Peripheral oxygen saturation (SpO2), a vital physiological sign employed in clinical care, is commonly obtained by using a contact pulse oximeter. With the rapid popularization of ordinary red-green-blue (RGB) webcams embedded in devices such as smartphones or laptops, there are broad application prospects for exploring techniques for non-contact SpO2 extraction using RGB webcams. However, many issues remain to be solved in the traditional webcam-based SpO2 extraction methods, such as the inherent low signal-to-noise ratio (SNR) of alternating current (AC) components of RGB signals and the potential defects in using RGB signals combination for SpO2 extraction. In this study, we conducted an in-depth examination of the existing research on webcam-based SpO2 extraction techniques, analyzed the practical problems in using them, and explored new ideas to solve the problems. Rather than roughly using the standard deviations (SD) of AC components for calculations, we performed blind source separation for AC components, and then used the energy coefficients retained in the mixed matrix to replace the variables required in the algorithm. Moreover, steady data was selected to compensate for the potential defects in using RGB signals combination. Through these efforts, the anti-noise capability of the algorithm was significantly enhanced, and the related defects were compensated for. The experimental results indicated that the proposed method produced reliable SpO2 estimation that could potentially-with further research-be used in real applications.

Multimodal imaging of laser speckle contrast imaging combined with mosaic filter-based hyperspectral imaging for precise surgical guidance

OBJECTIVE

To enable a real-time surgical guidance system that simultaneously monitors blood vessel perfusion, oxygen saturation, thrombosis, and tissue recovery by combining multiple optical imaging techniques into a single system: visible imaging, mosaic filter-based snapshot hyperspectral imaging (HSI), and laser speckle contrast imaging (LSCI).

METHODS

The multimodal optical imaging system was demonstrated by clamping blood vessels in the small intestines of rats to create areas of restricted blood flow. Subsequent tissue damage and regeneration were monitored during procedures. Using LSCI, vessel perfusion was measured, revealing the biological activity and survival of organ tissues. Blood oxygen saturation was monitored using HSI in the near-infrared region. Principal component analysis was used over the spectral dimension to identify an HSI wavelength combination optimized for hemodynamic biomarker visualization. HSI and LSCI were complimentary, identifying thrombus generation and tissue recovery, which was not possible in either modality alone.

RESULTS AND CONCLUSION

By analyzing multimodal tissue information from visible imaging, LSCI perfusion imaging, and HSI, a recovery prognosis could be determined based on the blood supply to the organ. The unique combination of the complementary imaging techniques into a single surgical microscope holds promise for improving the real-time determination of blood supply and tissue prognosis during surgery.

SIGNIFICANCE

Precise real-time monitoring for vascular anomalies promises to reduce the risk of organ damage in precise surgical operations such as tissue resection and transplantation. In addition, the convergence of label-free imaging technologies removes delays associated with the injection and diffusion of vascular monitoring dyes.

Infant heart-rate measurement and oxygen desaturation detection with a digital video camera using imaging photoplethysmography

OBJECTIVE

To assess the feasibility of using an ordinary digital video camera to measure heart rate and detect oxygen desaturations in healthy infants.

STUDY DESIGN

Heart rate and oxygen saturation were measured with a video camera by detecting small color changes in 28 infants' foreheads and compared with standard pulse oximetry measures. Multivariable regression examined the relationship between infant characteristics and heart-rate measurement precision.

RESULTS

The average bias of camera heart-rate measures was -4.2 beats per minute (BPM) and 95% limits of agreement were ±43.8 BPM. Desaturations detected by camera were 75% sensitive (15/20) and had a positive predictive value of 20% (15/74). Lower birth-weight was independently correlated with more precise heart-rate measures (8.05 BPM per kg, [95% CI 0.764-15.3]).

CONCLUSIONS

A digital video camera provides accurate but imprecise measures of infant heart rate and may provide a rough screening tool for oxygen desaturations.

Noncontact Monitoring of Heart Rate and Heart Rate Variability in Geriatric Patients Using Photoplethysmography Imaging

OBJECTIVE

Geriatric patients, especially those with dementia or in a delirious state, do not accept conventional contact-based monitoring. Therefore, we propose to measure heart rate (HR) and heart rate variability (HRV) of geriatric patients in a noncontact and unobtrusive way using photoplethysmography imaging (PPGI).

METHODS

PPGI video sequences were recorded from 10 geriatric patients and 10 healthy elderly people using a monochrome camera operating in the near-infrared spectrum and a colour camera operating in the visible spectrum. PPGI waveforms were extracted from both cameras using superpixel-based regions of interests (ROI). A classifier based on bagged trees was trained to automatically select artefact-free ROIs for HR estimation. HRV was calculated in the time-domain and frequency-domain.

RESULTS

an RMSE of 1.03 bpm and a correlation of 0.8 with the reference was achieved using the NIR camera for HR estimation. Using the RGB camera, RMSE and correlation improved to 0.48 bpm and 0.95, respectively. Correlation for HRV in the frequency-domain (LF/HF-ratio) was 0.50 using the NIR camera and 0.70 using the RGB camera.

CONCLUSION

We were able to demonstrate that PPGI is very suitable to measure HR and HRV in geriatric patients. We strongly believe that PPGI will become clinically relevant in monitoring of geriatric patients.

SIGNIFICANCE

we are the first group to measure both HR and HRV in awake geriatric patients using PPGI. Moreover, we systematically evaluate the effects of the spectrum (near-infrared vs. visible), ROI, and additional motion artefact reduction algorithms on the accuracy of estimated HR and HRV.

Non-Contact SpO2 Prediction System Based on a Digital Camera

Patients with the COVID-19 condition require frequent and accurate blood oxygen saturation (SpO2) monitoring. The existing pulse oximeters, however, require contact-based measurement using clips or otherwise fixed sensor units or need dedicated hardware which may cause inconvenience and involve additional appointments with the patient. This study proposes a computer vision-based system using a digital camera to measure SpO2 on the basis of the imaging photoplethysmography (iPPG) signal extracted from the human’s forehead without the need for restricting the subject or physical contact. The proposed camera-based system decomposes the iPPG obtained from the red and green channels into different signals with different frequencies using a signal decomposition technique based on a complete Ensemble Empirical Mode Decomposition (EEMD) technique and Independent Component Analysis (ICA) technique to obtain the optical properties from these wavelengths and frequency channels. The proposed system is convenient, contactless, safe and cost-effective. The preliminary results for 70 videos obtained from 14 subjects of different ages and with different skin tones showed that the red and green wavelengths could be used to estimate SpO2 with good agreement and low error ratio compared to the gold standard of pulse oximetry (SA210) with a fixed measurement position.

Reducing the effects of parallax in camera-based pulse-oximetry

Camera-based pulse-oximetry enables contactless estimation of peripheral oxygen saturation (SpO₂). Because of the lack of readily available and affordable single-optics multi-spectral cameras, custom-made multi-camera setups with different optical filters are currently mostly used. The introduced parallax by these cameras could however jeopardise the SpO₂ algorithm assumptions, especially during subject movement. In this paper we investigate the effect of parallax quantitatively by creating a large dataset consisting of 150 videos with three different parallax settings and with realistic and challenging motion scenarios. We estimate oxygen saturation values with a previously used global frame registration method and with a newly proposed adaptive local registration method to further reduce the parallax-induced image misalignment. We found that the amount of parallax has an important effect on the accuracy of the SpO₂ measurement during movement and that the proposed local image registration reduces the error by more than a factor of 2 for the most common motion scenarios during screening. Extrapolation of the results suggests that the error during the most challenging motion scenario can be reduced to approximately 2 percent when using a parallax-free single-optics camera. This study provides important insights on the possible applications and use cases of remote pulse-oximetry with current affordable and readily available cameras.

IoT Platform for COVID-19 Prevention and Control: A Survey

As a result of the worldwide transmission of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), coronavirus disease 2019 (COVID-19) has evolved into an unprecedented pandemic. Currently, with unavailable pharmaceutical treatments and low vaccination rates, this novel coronavirus results in a great impact on public health, human society, and global economy, which is likely to last for many years. One of the lessons learned from the COVID-19 pandemic is that a long-term system with non-pharmaceutical interventions for preventing and controlling new infectious diseases is desirable to be implemented. Internet of things (IoT) platform is preferred to be utilized to achieve this goal, due to its ubiquitous sensing ability and seamless connectivity. IoT technology is changing our lives through smart healthcare, smart home, and smart city, which aims to build a more convenient and intelligent community. This paper presents how the IoT could be incorporated into the epidemic prevention and control system. Specifically, we demonstrate a potential fog-cloud combined IoT platform that can be used in the systematic and intelligent COVID-19 prevention and control, which involves five interventions including COVID-19 Symptom Diagnosis, Quarantine Monitoring, Contact Tracing & Social Distancing, COVID-19 Outbreak Forecasting, and SARS-CoV-2 Mutation Tracking. We investigate and review the state-of-the-art literatures of these five interventions to present the capabilities of IoT in countering against the current COVID-19 pandemic or future infectious disease epidemics.

Robust non-contact peripheral oxygenation saturation measurement using smartphone-enabled imaging photoplethysmography

We propose a robust non-contact method to accurately estimate peripheral oxygen saturation (SpO₂) using a smartphone-based imaging photoplethysmography. The method utilizes the built-in color camera as a remote sensor and the built-in flashlight as illumination to estimate the SpO₂. Following the ratio of ratios between green and red channels, we introduce a multiple linear regression algorithm to improve the SpO₂ estimation. The algorithm considers the ratio of ratios and the reflectance images recorded at the RGB channels during a calibration process to obtain a set of weighting coefficients to weigh each contributor to the final determination of SpO₂. We demonstrate the proposed smartphone-based method of estimating the SpO₂ on five healthy volunteers whose arms are conditioned by a manual pressure cuff to manipulate the SpO₂ between 90∼100% as detected simultaneously by a medical-grade pulse oximeter. Experimental results indicate that the overall estimated error between the smartphone and the reference pulse oximeter is 0.029 ± 1.141%, leading to a 43% improvement over the conventional ratio of ratios method (0.008 ± 2.008%). In addition, the data sampling time in the current method is 2 seconds, similar to the sampling cycle used in the commercial medical-grade pulse oximeters.

Noncontact Sensing of Contagion

The World Health Organization (WHO) has declared COVID-19 a pandemic. We review and reduce the clinical literature on diagnosis of COVID-19 through symptoms that might be remotely detected as of early May 2020. Vital signs associated with respiratory distress and fever, coughing, and visible infections have been reported. Fever screening by temperature monitoring is currently popular. However, improved noncontact detection is sought. Vital signs including heart rate and respiratory rate are affected by the condition. Cough, fatigue, and visible infections are also reported as common symptoms. There are non-contact methods for measuring vital signs remotely that have been shown to have acceptable accuracy, reliability, and practicality in some settings. Each has its pros and cons and may perform well in some challenges but be inadequate in others. Our review shows that visible spectrum and thermal spectrum cameras offer the best options for truly noncontact sensing of those studied to date, thermal cameras due to their potential to measure all likely symptoms on a single camera, especially temperature, and video cameras due to their availability, cost, adaptability, and compatibility. Substantial supply chain disruptions during the pandemic and the widespread nature of the problem means that cost-effectiveness and availability are important considerations.

Detecting Pulse Wave From Unstable Facial Videos Recorded From Consumer-Level Cameras: A Disturbance-Adaptive Orthogonal Matching Pursuit

OBJECTIVE

Modern consumer-level cameras can detect subtle changes in human facial skin color due to varying blood flow; they are beginning to be used as noncontact devices to detect pulse waves. Little, however, do we know about their capacity to perform pulse wave detection when the recorded faces are unstable.

METHODS

Here, we propose a novel method that can extract pulse waves from videos with drastic facial unsteadiness such as head twists and alternating expressions. The method first uses chrominance characteristics in multiple facial sub-regions to construct a raw pulse matrix. Subsequently, it employs a disturbance-adaptive orthogonal matching pursuit (DAOMP) algorithm to recover the underlying pulse matrix corrupted by facial unsteadiness.

RESULTS

To evaluate the efficacy of the method, we perform analyses on two datasets including 268 samples from 67 testing subjects. The results demonstrate that the proposed method outperforms state-of-the-art algorithms, especially in the terrain where drastic facial unsteadiness is present.

CONCLUSION

The proposed framework shows promise to achieve videos-based noncontact pulse wave detection from both steady and unsteady faces recorded by consumer-level cameras.

SIGNIFICANCE

By employing the proposed method, disturbance robustness in noncontact pulse wave detection can be significantly improved.

Comparison of Visible and Infrared Video Plethysmography Captured from Different Regions of the Human Face

Recently, video plethysmography (VPG) - a heart rate estimation technique using a video camera - has gained significant attention. Most studies of VPG have used a visible RGB camera; only a limited number of studies investigating near-infrared light (wavelength 750-2500 nm), which can be used even in a dark environment, have been performed. The purpose of this study was to investigate the differences between VPG data collected using visible light (VPG_VIS) or near-infrared light (VPG_NIR) from four facial areas (forehead, right cheek, left cheek, and nose). An experiment was conducted to obtain both VPG_VIS and VPG_NIR simultaneously by alternately irradiating the face with NIR and VIS lights. Experimental results showed that the root mean squared error of heart rate estimated using VPG_NIR was 1 bpm higher than that of VPG_VIS. However, contrary to our expectations, the power of the heartbeat-related component included in VPG_NIR was not reduced despite the absorbance of hemoglobin in the NIR light range being 1/100 of that in the VIS light range. This result supports the hypothesis that a main factor in the generation of VPG waves was change in the optical properties caused by blood vessels compressing the subcutaneous tissue and the venous bed. Additionally, the accuracy of the heart rate estimation using VPG tended to be high when the nose was set as the ROI. This result was likely associated with the anatomical structure of the nose.

A camera-based ballistocardiogram heart rate measurement method

Recent studies have shown that head movements associated with cardiac activity contain a heart rate (HR) signal. In most previous studies, subjects were required to remain stationary in a specific environment during HR measurements, and measurement accuracy depended on the choice of target in the scene, i.e., the specified region of the face. In this paper, we proposed a robust HR measurement method based on ballistocardiogram (BCG) technology. This method requires only a camera and does not require that users establish a complex measurement environment. In addition, a bidirectional optical flow algorithm is designed to select and track valid feature points in the video captured by using the camera. Experiments with 11 subjects show that the HR values measured using the proposed method differ slightly from the reference values, and the average error is only 1.09%. Overall, this method can improve the accuracy of BCG without limitations related to skin tone, illumination, the state of the subject, or the test location.

Using Non-Contact Imaging Photoplethysmography to Recover Diurnal Patterns in Heart Rate

Daily patterns in cardiovascular signals can reveal important information about physiological processes, health and well-being. Traditionally, contact sensors have been used to collect longitudinal data of this kind. However, recent advances in non-contact imaging techniques have led to algorithms that can be used to measure vital signs unobtrusively. Imaging methods are highly scalable due to the availability of webcams and computing devices making them attractive for longitudinal, in-situ measurement. Using a software tool we captured over 1,000 hours of non-contact heart rate measurements, via imaging photoplethysmography. Using these data we were able to recover diurnal patterns in heart rate during the working day. Non-contact sensing techniques hold much promise but also raise ethical issues that need to be addressed seriously within the biomedical engineering community.

Data-Driven Calibration Estimation for Robust Remote Pulse-Oximetry

Pulse-oximetry has become a core monitoring modality in most fields of medicine. Typical dual-wavelength pulse-oximeters estimate blood oxygen saturation (SpO2) levels from a relationship between the amplitudes of red and infrared photoplethysmographic (PPG) waveforms. When captured with a camera, the PPG waveforms are much weaker and consequently the measurement is more sensitive to distortions and noises. Therefore, an indirect method has recently been proposed where, instead of extracting the relative amplitudes from the individual waveforms, the waveforms are linearly combined to construct a collection of pulse signals with different pulse signatures, each corresponding to a specific oxygen saturation level. This method has been shown to outperform the conventional ratio-of-ratios based methods, especially when adding a third wavelength. Adding wavelengths, however, complicates the calibration. Inaccuracies in the calibration model threaten the performance of the method. Opto-physiological models have been shown earlier to provide useful calibration parameter estimates. In this paper, we show that the accuracy can be improved using a data-driven approach. We performed 5-fold cross validation on recordings with variations in oxygen saturation and optimized for pulse quality. All evaluated wavelength combinations, also without visible red, meet the required ISO standard accuracy with the calibration from the proposed method. This scalable approach is not only helpful to fine-tune the calibration model, but even allows computation of the calibration model parameters from scratch without prior knowledge of the data acquisition details, i.e., the properties of camera and illumination.

Availability and performance of image-based, non-contact methods of monitoring heart rate, blood pressure, respiratory rate, and oxygen saturation: a systematic review

OBJECTIVE

Over the last 15 years, developments in camera technology have coincided with increased availability and affordability. This has led to an increasing interest in using these technologies in healthcare settings. Image-based monitoring methods potentially allow multiple vital signs to be measured concurrently using a non-contact sensor. We have undertaken a systematic review of the current availability and performance of these monitoring methods.

APPROACH

A multiple database search was conducted using MEDLINE, Embase, CINAHL, Cochrane Library, OpenGrey, IEEE Xplore Library and ACM Digital Library to July 2018. We included studies comparing image-based heart rate, respiratory rate, oxygen saturation and blood pressure monitoring methods against one or more validated reference device(s). Each included study was assessed using the modified GRRAS criteria for reporting bias.

MAIN RESULTS

Of 30 279 identified studies, 161 were included in the final analysis. Twenty studies (20/161, 12%) were carried out on patients in clinical settings, while the remainder were conducted in academic settings using healthy volunteer populations. The 18-40 age group was best represented across the identified studies. One hundred and twenty studies (120/161, 75%) estimated heart rate, followed by 62 studies (62/161, 39%) estimating respiratory rate. Fewer studies focused on oxygen saturation (11/161, 7%) or blood pressure (6/161, 4%) estimation. Fifty-one heart rate studies (51/120, 43%) and 24 respiratory rate studies (24/62, 39%) used Bland-Altman analysis to report their results. Of the heart rate studies, 28 studies (28/51, 55%) showed agreement within industry standards of [Formula: see text]5 beats per minute. Only two studies achieved this within clinical settings. Of the respiratory rate studies, 13 studies (13/24, 54%) showed agreement within industry standards of [Formula: see text]3 breaths per minute, but only one study achieved this in a clinical setting. Statistical analysis was heterogeneous across studies with frequent inappropriate use of correlation. The majority of studies (99/161, 61%) monitored subjects for under 5 min. Three studies (3/161, 2%) monitored subjects for over 60 min, all of which were conducted in hospital settings.

SIGNIFICANCE

Heart rate and respiratory rate monitoring using video images is currently possible and performs within clinically acceptable limits under experimental conditions. Camera-derived estimates were less accurate in the proportion of studies conducted in clinical settings. We would encourage thorough reporting of the population studied, details of clinically relevant aspects of methodology, and the use of appropriate statistical methods in future studies. Systematic review registration: PROSPERO CRD42016029167 Protocol: https://systematicreviewsjournal.biomedcentral.com/articles/10.1186/s13643-017-0615-3.

Cardiovascular assessment by imaging photoplethysmography - a review

Over the last few years, the contactless acquisition of cardiovascular parameters using cameras has gained immense attention. The technique provides an optical means to acquire cardiovascular information in a very convenient way. This review provides an overview on the technique's background and current realizations. Besides giving detailed information on the most widespread application of the technique, namely the contactless acquisition of heart rate, we outline further concepts and we critically discuss the current state.

Use of Mobile Applications to Collect Data in Sport, Health, and Exercise Science: A Narrative Review

Peart, DJ, Balsalobre-Fernández, C, and Shaw, MP. Use of mobile applications to collect data in sport, health, and exercise science: A narrative review. J Strength Cond Res 33(4): 1167-1177, 2019-Mobile devices are ubiquitous in the population, and most have the capacity to download applications (apps). Some apps have been developed to collect physiological, kinanthropometric, and performance data; however, the validity and reliability of such data is often unknown. An appraisal of such apps is warranted, as mobile apps may offer an alternative method of data collection for practitioners and athletes with money, time, and space constraints. This article identifies and critically reviews the commercially available apps that have been tested in the scientific literature, finding evidence to support the measurement of the resting heart through photoplethysmography, heart rate variability, range of motion, barbell velocity, vertical jump, mechanical variables during running, and distances covered during walking, jogging, and running. The specific apps with evidence, along with reported measurement errors are summarized in the review. Although mobile apps may have the potential to collect data in the field, athletes and practitioners should exercise caution when implementing them into practice as not all apps have support from the literature, and the performance of a number of apps have only been tested on 1 device.

Contact-Less Real-Time Monitoring of Cardiovascular Risk Using Video Imaging and Fuzzy Inference Rules

Conventional methods for measuring cardiovascular parameters use skin contact techniques requiring a measuring device to be worn by the user. To avoid discomfort of contact devices, camera-based techniques using photoplethysmography have been recently introduced. Nevertheless, these solutions are typically expensive and difficult to be used daily at home. In this work, we propose an innovative solution for monitoring cardiovascular parameters that is low cost and can be easily integrated within any common home environment. The proposed system is a contact-less device composed of a see-through mirror equipped with a camera that detects the person’s face and processes video frames using photoplethysmography in order to estimate the heart rate, the breath rate and the blood oxygen saturation. In addition, the color of lips is automatically detected via clustering-based color quantization. The estimated parameters are used to predict a risk of cardiovascular disease by means of fuzzy inference rules integrated in the mirror-based monitoring system. Comparing our system to a contact device in measuring vital parameters on still or slightly moving subjects, we achieve measurement errors that are within acceptable margins according to the literature. Moreover, in most cases, the response of the fuzzy rule-based system is comparable with that of the clinician in assessing a risk level of cardiovascular disease.

Measuring Oxygen Saturation With Smartphone Cameras Using Convolutional Neural Networks

Arterial oxygen saturation ([Formula: see text]) is an indicator of how much oxygen is carried by hemoglobin in the blood. Having enough oxygen is vital for the functioning of cells in the human body. Measurement of [Formula: see text] is typically estimated with a pulse oximeter, but recent works have investigated how smartphone cameras can be used to infer [Formula: see text]. In this paper, we propose methods for the measurement of [Formula: see text] with a smartphone using convolutional neural networks and preprocessing steps to better guard against motion artifacts. To evaluate this methodology, we conducted a breath-holding study involving 39 participants. We compare the results using two different mobile phones. We compare our model with the ratio-of-ratios model that is widely used in pulse oximeter applications, showing that our system has significantly lower mean absolute error (2.02%) than a medical pulse oximeter.

Optical Fibre-Based Pulse Oximetry Sensor with Contact Force Detection

A novel optical sensor probe combining monitoring of blood oxygen saturation (SpO₂) with contact pressure is presented. This is beneficial as contact pressure is known to affect SpO₂ measurement. The sensor consists of three plastic optical fibres (POF) used to deliver and collect light for pulse oximetry, and a fibre Bragg grating (FBG) sensor to measure contact pressure. All optical fibres are housed in a biocompatible epoxy patch which serves two purposes: (i) to reduce motion artefacts in the photoplethysmogram (PPG), and (ii) to transduce transverse loading into an axial strain in the FBG. Test results show that using a combination of pressure measuring FBG with a reference FBG, reliable results are possible with low hysteresis which are relatively immune to the effects of temperature. The sensor is used to measure the SpO₂ of ten volunteers under different contact pressures with perfusion and skewness indices applied to assess the quality of the PPG. The study revealed that the contact force ranging from 5 to 15 kPa provides errors of <2%. The combined probe has the potential to improve the reliability of reflectance oximeters. In particular, in wearable technology, the probe should find use in optimising the fitting of garments incorporating this technology.