Reducing the effects of parallax in camera-based pulse-oximetry

Leveraging 3D convolutional neural network and 3D visible-near-infrared multimodal imaging for enhanced contactless oximetry

Significance

Monitoring oxygen saturation (SpO 2 ) is important in healthcare, especially for diagnosing and managing pulmonary diseases. Non-contact approaches broaden the potential applications ofSpO 2 measurement by better hygiene, comfort, and capability for long-term monitoring. However, existing studies often encounter challenges such as lower signal-to-noise ratios and stringent environmental conditions.

Aim

We aim to develop and validate a contactlessSpO 2 measurement approach using 3D convolutional neural networks (3D CNN) and 3D visible-near-infrared (VIS-NIR) multimodal imaging, to offer a convenient, accurate, and robust alternative forSpO 2 monitoring.

Approach

We propose an approach that utilizes a 3D VIS-NIR multimodal camera system to capture facial videos, in whichSpO 2 is estimated through 3D CNN by simultaneously extracting spatial and temporal features. Our approach includes registration of multimodal images, tracking of the 3D region of interest, spatial and temporal preprocessing, and 3D CNN-based feature extraction andSpO 2 regression.

Results

In a breath-holding experiment involving 23 healthy participants, we obtained multimodal video data with referenceSpO 2 values ranging from 80% to 99% measured by pulse oximeter on the fingertip. The approach achieved a mean absolute error (MAE) of 2.31% and a Pearson correlation coefficient of 0.64 in the experiment, demonstrating good agreement with traditional pulse oximetry. The discrepancy of estimatedSpO 2 values was within 3% of the referenceSpO 2 for ∼ 80 % of all 1-s time points. Besides, in clinical trials involving patients with sleep apnea syndrome, our approach demonstrated robust performance, with an MAE of less than 2% inSpO 2 estimations compared to gold-standard polysomnography.

Conclusions

The proposed approach offers a promising alternative for non-contact oxygen saturation measurement with good sensitivity to desaturation, showing potential for applications in clinical settings.

Notch RGB-camera based SpO2 estimation: a clinical trial in a neonatal intensive care unit

Regular and narrow-band RGB cameras are recently explored for contactless SpO2 monitoring. Regular RGB cameras with cross-band overlap provide a high signal-to-noise-ratio (SNR) in measuring the photoplethysmographic signals but possess high dependency on the spectra of incident light, whereas narrow-band RGB cameras have better spectral independence but lower SNR especially in dim lighting conditions, such as in the neonatal intensive care unit (NICU). This paper proposes a notch RGB camera based SpO2 measurement approach that uses an optical notch filter to attenuate the wavelengths of 580-605 nm of a regular RGB camera to improve the spectral independence while maintaining high SNR in signal measurement. The proposed setup was validated in the lab condition (e.g. dark chamber) against the existing solutions for visible-light based camera-SpO2 measurement and further verified in the NICU on preterm infants. The clinical trial conducted in the NICU with 22 preterm infants shows that the notch RGB camera can achieve a mean absolute error (MAE) less than 4% for SpO2 measurement. This is the first showcase of continuous monitoring of absolute camera-SpO2 values in the NICU.

Impact of Different Skin Penetration Depths of Red and Green Wavelengths on Camera-based SpO2 Measurement

Remote camera-based estimation of blood oxygen saturation (SpO2) using visible lights has been studied recently, typically for red (660 nm) and green (550 nm) wavelengths. This paper investigates the impact of different skin penetration depths of red and green wavelengths on the SpO2 estimation based on mathematical modeling and experiments, where the SpO2-calibritability between two illumination setups, narrow-band red/green and narrow-band red/infrared, are statistically compared using the "ratio-of-ratios" method. The results show that the performance of the setup using red/green is less consistent among 17 volunteers than the setup using red/infrared, and larger SpO2 disparity between different skin regions (by SpO2 imaging) have been found for individuals in the red/green wavelengths setup. The use of visible light (red and green) may impose a risk of SpO2 calibration due to the different skin penetration depths of these two wavelengths.

Contactless SpO2 with an RGB camera: experimental proof of calibrated SpO2

Camera-based blood oxygen saturation (SpO2) monitoring allows reliable measurements without touching the skin and is therefore very attractive when there is a risk of cross-infection, in case of fragile skin, and/or to improve the clinical workflow. Despite promising results, productization of the technology is hampered by the unavailability of adequate hardware, especially a camera, which can capture the optimal wavelengths for SpO2 measurements in the red near-infrared region. A regular color (RGB) camera is attractive because of its availability, but also poses several risks and challenges which affect the accuracy of the measurement. To mitigate the most important risks, we propose to add low-cost commercial off-the-shelf (COTS) components to the setup. We executed two studies with this setup: one at a hypoxia lab with SpO2 values in the range 70 - 100% with the purpose to determine the calibration model, and the other study on volunteers to investigate the accuracy for different spot-check scenarios. The proposed processing pipeline includes face tracking and a robust method to estimate the ratio of relative amplitudes of the photoplethysmographic waveforms. Results show that the error is smaller than 4 percent points for realistic screening scenarios where the subject is seated, either with or without head support and/or ambient light.

Photoplethysmographic Imaging of Hemodynamics and Two-Dimensional Oximetry

The review of recent papers devoted to actively developing methods of photoplethysmographic imaging (the PPGI) of blood volume pulsations in vessels and non-contact two-dimensional oximetry on the surface of a human body has been carried out. The physical fundamentals and technical aspects of the PPGI and oximetry have been considered. The manifold of the physiological parameters available for the analysis by the PPGI method has been shown. The prospects of the PPGI technology have been discussed. The possibilities of non-contact determination of blood oxygen saturation SpO2 (pulse saturation O2) have been described. The relevance of remote determination of the level of oxygenation in connection with the spread of a new coronavirus infection SARS-CoV-2 (COVID-19) has been emphasized. Most of the works under consideration cover the period 2010-2021.

Contactless Vital Signs Acquisition Using Video Photoplethysmography, Motion Analysis and Passive Infrared Thermography Devices During Emergency Department Walk-In Triage in Pandemic Conditions

BACKGROUND

Contactless vital signs (VS) measurement with video photoplethysmography (vPPG), motion analysis (MA), and passive infrared thermometry (pIR) has shown promise.

OBJECTIVES

To compare conventional (contact-based) and experimental contactless VS measurement approaches for emergency department (ED) walk-in triage in pandemic conditions.

METHODS

Patients' heart rates (HR), respiratory rates (RR), and temperatures were measured with cardiorespiratory monitor and vPPG, manual count and MA, and contact thermometers and pIR, respectively.

RESULTS

There were 475 walk-in ED patients studied (95% of eligible). Subjects were 35.2 ± 20.8 years old (range 4 days‒95 years); 52% female, 0.2% transgender; had Fitzpatrick skin type of 2.3 ± 1.4 (range 1‒6), Emergency Severity Index of 3.0 ± 0.6 (range 2‒5), and contact temperature of 36.83°C (range 35.89-39.4°C) (98.3°F [96.6‒103°F]). Pediatric HR and RR data were excluded from analysis due to research challenges associated with pandemic workflow. For a 30-s, unprimed "Triage" window in 377 adult patients, vPPG-MA acquired 377 (100%) HR measurements featuring a mean difference with cardiorespiratory monitor HR of 5.9 ± 12.8 beats/min (R = 0.6833) and 252 (66.8%) RR measurements featuring a mean difference with manual RR of -0.4 ± 2.6 beats/min (R = 0.8128). Subjects' Emergency Severity Index components based on conventional VS and contactless VS matched for 83.8% (HR) and 89.3% (RR). Filtering out vPPG-MA measurements with low algorithmic confidence reduced VS acquired while improving correlation with conventional measurements. The mean difference between contact and pIR temperatures was 0.83 ± 0.67°C (range -1.16-3.5°C) (1.5 ± 1.2°F [range -2.1-6.3°F]); pIR fever detection improved with post hoc adjustment for mean bias.

CONCLUSION

Contactless VS acquisition demonstrated good agreement with contact methods during adult walk-in ED patient triage in pandemic conditions; clinical applications will need further study.