Algorithmic insights of camera-based respiratory motion extraction

Privacy-Protected Contactless Sleep Parameters Measurement Using a Defocused Camera

Sleep monitoring plays a vital role in various scenarios such as hospitals and living-assisted homes, contributing to the prevention of sleep accidents as well as the assessment of sleep health. Contactless camera-based sleep monitoring is promising due to its user-friendly nature and rich visual semantics. However, the privacy concern of video cameras limits their applications in sleep monitoring. In this paper, we explored the opportunity of using a defocused camera that does not allow identification of the monitored subject when measuring sleep-related parameters, as face detection and recognition are impossible on optically blurred images. We proposed a novel privacy-protected sleep parameters measurement framework, including a physiological measurement branch and a semantic analysis branch based on ResNet-18. Four important sleep parameters are measured: heart rate (HR), respiration rate (RR), sleep posture, and movement. The results of HR, RR, and movement have strong correlations with the reference (HR: R = 0.9076; RR: R = 0.9734; Movement: R = 0.9946). The overall mean absolute errors (MAE) for HR and RR are 5.2 bpm and 1.5 bpm respectively. The measurement of HR and RR achieve reliable estimation coverage of 72.1% and 93.6%, respectively. The sleep posture detection achieves an overall accuracy of 94.5%. Experimental results show that the defocused camera is promising for sleep monitoring as it fundamentally eliminates the privacy issue while still allowing the measurement of multiple parameters that are essential for sleep health informatics.

Multispectral Depolarization Based Living-Skin Detection: A New Measurement Principle

Camera-based photoplethysmographic imaging enabled the segmentation of living-skin tissues in a video, but it has inherent limitations to be used in real-life applications such as video health monitoring and face anti-spoofing. Inspired by the use of polarization for improving vital signs monitoring (i.e. specular reflection removal), we observed that skin tissues have an attractive property of wavelength-dependent depolarization due to its multi-layer structure containing different absorbing chromophores, i.e. polarized light photons with longer wavelengths (R) have deeper skin penetrability and thus experience thorougher depolarization than those with shorter wavelengths (G and B). Thus we proposed a novel dual-polarization setup and an elegant algorithm (named "MSD") that exploits the nature of multispectral depolarization of skin tissues to detect living-skin pixels, which only requires two images sampled at the parallel and cross polarizations to estimate the characteristic chromaticity changes (R/G) caused by tissue depolarization. Our proposal was verified in both the laboratory and hospital settings (ICU and NICU) focused on anti-spoofing and patient skin segmentation. The clinical experiments in ICU also indicate the potential of MSD for skin perfusion analysis, which may lead to a new diagnostic imaging approach in the future.

On the cardiorespiratory coordination assessed by the photoplethysmography imaging technique

Cardiorespiratory coordination (CRC) probes the interaction between cardiac and respiratory oscillators in which cardiac and respiratory activity are synchronized, with individual heartbeats occurring at approximately the same temporal positions during several breathing cycles. An increase of CRC has previously been related to pathological stressful states. We studied CRC employing coordigrams computed from non-contact photoplethysmography imaging (PPGI) and respiratory data using the optical flow method. In a blocked study design, we applied the cold pressure test (CPT), water at ambient temperature (AWT), and intermittent resting conditions. In controls (no intervention), CRC remained on initial low levels throughout measurements. In the experimental group (AWT and CPT intervention), CRC decreased during AWT and CPT. Following both interventions, CRC increased significantly, with a rebound effect following AWT. In controls, HR increased steadily over time. CPT evoked a significant HR increase which correlated with subjective stress/pain ratings. The CRC increase following AWT correlated significantly with subjective pain (r = .79) and stress (r = .63) ratings. Furthermore, we observed a significant correlation (r = - .80) between mean RMSSD and mean duration of CRC, which further supports an association between autonomic state and CRC level. CRC analysis obtained from cutaneous tissue perfusion data therefore appears to be a sensitive and useful method for the study of CRC and ANS activity. Future studies need to investigate the physiological principles and clinical significance of these findings.

Camera-based Respiratory Imaging for Thoracic Asymmetry in Thoracic Surgery Patients

The current tool for assessing thoracic asymmetry of thoracic surgery patients is inappropriate for timely or frequent clinical routines due to its dependency on empirical physical examinations or specialized machines. This study investigates the camera-based respiratory imaging for screening thoracic asymmetry, in an intelligent and convenient way. The respiratory heatmaps are generated based on the respiratory magnitudes, phases and angles extracted from the chest video, and bilateral chest region of interest are compared statistically. Due to the variability of chest respiratory direction, spatial enhancement (SDR and SPCA) algorithms are proposed to magnify the respiratory energy. The proposed framework was validated in a clinical trial involving 31 patients, recorded by a smartphone camera. A high correlation was found between the camera measurements and patients' thoracic status in both the visual imaging and quantified indices. The respiratory imaging of camera shows a clear potential for assessing chest abnormalities of thoracic surgery patients.

Occlusion-robust Sleep Posture Detection using Body Rolling Motion in a Video

It has been reported that the monitoring of sleep postures is useful for the treatment and prevention of sleep diseases such as obstructive sleep apnea and heart failure. Camera-based sleep posture detection is attractive for the nature of comfort and convenience of use. However, the main challenge is to detect postures from images of the body that are occluded by bed sheets or covers. To address this issue, we propose a novel occlusion-robust sleep posture detection method exploiting the body rolling motion in a video. It uses the head orientation to indicate the posture direction (supine, left or right lateral), triggered by the full-body rolling motion (as a sign of posture change). The experimental results show that our proposed method, as compared with the state-of-the-art approaches such as skeleton-based (MediaPipe) and full-image ResNet based methods, obtained clear improvements on sleep posture detection with heavy body occlusions, with an averaged precision, recall and F1-score of 0.974, 0.993 and 0.983, respectively. The next step is to integrate the sleep posture detection algorithm into a camera-based sleep monitoring system for clinical validations.

Time-domain Features of Angular-velocity Signals for Camera-based Respiratory RoI detection: A Clinical Study in NICU

Camera-based measurement of respiratory rate (RR) is emerging for preterm infants monitoring in Neonatal Intensive Care Units (NICU). Accurate detection of respiratory region of interest (Resp-RoI), e.g. thorax and abdomen of infants, is essential for achieving a fully-automatic solution and for high-quality RR estimation. However, the application of fast Fourier transform (FFT) for detecting Resp-RoI in premature infants may not be appropriate due to their irregular breathing patterns. This study proposes a new method for detecting Resp-RoIs in premature infants that uses time-domain features of angular-velocity of respiration. By fusing respiratory motion on orthogonal directions, the proposed method is more robust to variations of infant posture in the incubator.. In addition, using inter-beat interval (IBI) features in the time domain helps to distinguish between Resp-RoI and background. The proposed method was validated on 20 preterm infants in NICU. It obtains a clear improvement on Resp-RoI detection (RoI correspondence = 0.74) and RR estimation (MAE = 3.62 bpm) against the benchmarked approaches (maxFFT: RoI correspondence = 0.45, MAE = 5.61 bpm).

Feasibility of Exploiting Physiological and Motion Features for Camera-based Sleep Staging: A Clinical Study

Camera-based sleep monitoring is an emergent research topic in sleep medicine. The feasibility of using both the physiological features and motion features measured by a video camera for sleep staging was not thoroughly investigated. In this paper, we built a camera-based non-contact sleep monitoring setup in the Institute of Respiratory Diseases, Shenzhen People's Hospital, and created a clinical sleep dataset (nocturnal video data of 11 adults) including the expert-corrected PSG references synchronized with the video. The camera-based measurements have shown high correlations with the PSG. It obtains an overall Mean Absolute Error (MAE) of 1.5 bpm for heart-rate (HR), 0.7 bpm for breathing-rate (BR), 13.9 ms for heart-rate variability (HRV), and an accuracy of 93.5% for leg motion detection. The statistical analysis indicates that the averaged HR and variations of BR are distinct features for annotating four sleep stages (awake, REM, light sleep, and deep sleep). HRV parameter (SDNN) can clearly differentiate rapid-eye-movement (REM) and non-REM, while the leg movement is a distinctive feature for separating awake and sleep. The clinical trial demonstrated the feasibility of using physiological and motion features measured by a camera for joint sleep staging, and provides insights for sleep-related feature selection.