Respiratory Rate Estimation using PPG: A Deep Learning Approach

Contactless Blood Oxygen Saturation Estimation from Facial Videos Using Deep Learning

Blood oxygen saturation (SpO2) is an essential physiological parameter for evaluating a person's health. While conventional SpO2 measurement devices like pulse oximeters require skin contact, advanced computer vision technology can enable remote SpO2 monitoring through a regular camera without skin contact. In this paper, we propose novel deep learning models to measure SpO2 remotely from facial videos and evaluate them using a public benchmark database, VIPL-HR. We utilize a spatial-temporal representation to encode SpO2 information recorded by conventional RGB cameras and directly pass it into selected convolutional neural networks to predict SpO2. The best deep learning model achieves 1.274% in mean absolute error and 1.71% in root mean squared error, which exceed the international standard of 4% for an approved pulse oximeter. Our results significantly outperform the conventional analytical Ratio of Ratios model for contactless SpO2 measurement. Results of sensitivity analyses of the influence of spatial-temporal representation color spaces, subject scenarios, acquisition devices, and SpO2 ranges on the model performance are reported with explainability analyses to provide more insights for this emerging research field.

Advances in Respiratory Monitoring: A Comprehensive Review of Wearable and Remote Technologies

This article explores the importance of wearable and remote technologies in healthcare. The focus highlights its potential in continuous monitoring, examines the specificity of the issue, and offers a view of proactive healthcare. Our research describes a wide range of device types and scientific methodologies, starting from traditional chest belts to their modern alternatives and cutting-edge bioamplifiers that distinguish breathing from chest impedance variations. We also investigated innovative technologies such as the monitoring of thorax micromovements based on the principles of seismocardiography, ballistocardiography, remote camera recordings, deployment of integrated optical fibers, or extraction of respiration from cardiovascular variables. Our review is extended to include acoustic methods and breath and blood gas analysis, providing a comprehensive overview of different approaches to respiratory monitoring. The topic of monitoring respiration with wearable and remote electronics is currently the center of attention of researchers, which is also reflected by the growing number of publications. In our manuscript, we offer an overview of the most interesting ones.

A Novel Respiratory Rate Estimation Algorithm from Photoplethysmogram Using Deep Learning Model

Respiratory rate (RR) is a critical vital sign that can provide valuable insights into various medical conditions, including pneumonia. Unfortunately, manual RR counting is often unreliable and discontinuous. Current RR estimation algorithms either lack the necessary accuracy or demand extensive window sizes. In response to these challenges, this study introduces a novel method for continuously estimating RR from photoplethysmogram (PPG) with a reduced window size and lower processing requirements. To evaluate and compare classical and deep learning algorithms, this study leverages the BIDMC and CapnoBase datasets, employing the Respiratory Rate Estimation (RRest) toolbox. The optimal classical techniques combination on the BIDMC datasets achieves a mean absolute error (MAE) of 1.9 breaths/min. Additionally, the developed neural network model utilises convolutional and long short-term memory layers to estimate RR effectively. The best-performing model, with a 50% train-test split and a window size of 7 s, achieves an MAE of 2 breaths/min. Furthermore, compared to other deep learning algorithms with window sizes of 16, 32, and 64 s, this study's model demonstrates superior performance with a smaller window size. The study suggests that further research into more precise signal processing techniques may enhance RR estimation from PPG signals.

Evaluation of the Photoplethysmogram-Based Deep Learning Model for Continuous Respiratory Rate Estimation in Surgical Intensive Care Unit

The respiratory rate (RR) is a significant indicator to evaluate a patient's prognosis and status; however, it requires specific instrumentation or estimates from other monitored signals. A photoplethysmogram (PPG) is extensively used in clinical environments as well as in intensive care units (ICUs) to primarily monitor peripheral circulation while capturing indirect information about intrathoracic pressure changes. This study aims to apply and evaluate several deep learning models using a PPG for the continuous and accurate estimation of the RRs of patients. The dataset was collected twice for 2 min each in 100 patients aged 18 years and older from the surgical intensive care unit of a tertiary referral hospital. The BIDMC and CapnoBase public datasets were also analyzed. The collected dataset was preprocessed and split according to the 5-fold cross-validation. We used seven deep learning models, including our own Dilated Residual Neural Network, to check how accurately the RR estimates match the ground truth using the mean absolute error (MAE). As a result, when validated using the collected dataset, our model showed the best results with a 1.2628 ± 0.2697 MAE on BIDMC and RespNet and with a 3.1268 ± 0.6363 MAE on our dataset, respectively. In conclusion, RR estimation using PPG-derived models is still challenging and has many limitations. However, if there is an equal amount of data from various breathing groups to train, we expect that various models, including our Dilated ResNet model, which showed good results, can achieve better results than the current ones.

Predicting Respiratory Rate from Electrocardiogram and Photoplethysmogram Using a Transformer-Based Model

The respiratory rate (RR) serves as a critical physiological parameter in the context of both diagnostic and prognostic evaluations. Due to the challenges of direct measurement, RR is still predominantly measured through the traditional manual counting-breaths method in clinic practice. Numerous algorithms and machine learning models have been developed to predict RR using physiological signals, such as electrocardiogram (ECG) or/and photoplethysmogram (PPG) signals. Yet, the accuracy of these existing methods on available datasets remains limited, and their prediction on new data is also unsatisfactory for actual clinical applications. In this paper, we proposed an enhanced Transformer model with inception blocks for predicting RR based on both ECG and PPG signals. To evaluate the generalization capability on new data, our model was trained and tested using subject-level ten-fold cross-validation using data from both BIDMC and CapnoBase datasets. On the test set, our model achieved superior performance over five popular deep-learning-based methods with mean absolute error (1.2) decreased by 36.5% and correlation coefficient (0.85) increased by 84.8% compared to the best results of these models. In addition, we also proposed a new pipeline to preprocess ECG and PPG signals to improve model performance. We believe that the development of the TransRR model is expected to further expedite the clinical implementation of automatic RR estimation.

Whale Optimization Algorithm with a Hybrid Relation Vector Machine: A Highly Robust Respiratory Rate Prediction Model Using Photoplethysmography Signals

Due to the simplicity and convenience of PPG signal acquisition, the detection of the respiration rate based on the PPG signal is more suitable for dynamic monitoring than the impedance spirometry method, but it is challenging to achieve accurate predictions from low-signal-quality PPG signals, especially in intensive-care patients with weak PPG signals. The goal of this study was to construct a simple model for respiration rate estimation based on PPG signals using a machine-learning approach fusing signal quality metrics to improve the accuracy of estimation despite the low-signal-quality PPG signals. In this study, we propose a method based on the whale optimization algorithm (WOA) with a hybrid relation vector machine (HRVM) to construct a highly robust model considering signal quality factors to estimate RR from PPG signals in real time. To detect the performance of the proposed model, we simultaneously recorded PPG signals and impedance respiratory rates obtained from the BIDMC dataset. The results of the respiration rate prediction model proposed in this study showed that the MAE and RMSE were 0.71 and 0.99 breaths/min, respectively, in the training set, and 1.24 and 1.79 breaths/min, respectively, in the test set. Compared without taking signal quality factors into account, MAE and RMSE are reduced by 1.28 and 1.67 breaths/min, respectively, in the training set, and reduced by 0.62 and 0.65 breaths/min in the test set. Even in the nonnormal breathing range below 12 bpm and above 24 bpm, the MAE reached 2.68 and 4.28 breaths/min, respectively, and the RMSE reached 3.52 and 5.01 breaths/min, respectively. The results show that the model that considers the PPG signal quality and respiratory quality proposed in this study has obvious advantages and application potential in predicting the respiration rate to cope with the problem of low signal quality.

Using self-supervised feature learning to improve the use of pulse oximeter signals to predict paediatric hospitalization

Background: The success of many machine learning applications depends on knowledge about the relationship between the input data and the task of interest (output), hindering the application of machine learning to novel tasks. End-to-end deep learning, which does not require intermediate feature engineering, has been recommended to overcome this challenge but end-to-end deep learning models require large labelled training data sets often unavailable in many medical applications. In this study, we trained self-supervised learning (SSL) models for automatic feature extraction from raw photoplethysmography (PPG) obtained using a pulse oximeter, with the aim of predicting paediatric hospitalization.  Methods: We compared logistic regression models fitted using features extracted using SSL with models trained using both clinical and SSL features. In addition, we compared end-to-end deep learning models initialized randomly or using weights from the SSL models. We also compared the performance of SSL models trained on labelled data alone (n=1,031) with SSL trained using both labelled and unlabelled signals (n=7,578). Results: Logistic regression models were more predictive of hospitalization when trained on features extracted using labelled PPG signals only compared to SSL models trained on both labelled and unlabelled signals (AUC 0.83 vs 0.80). However, features extracted using SSL model trained on both labelled and unlabelled PPG signals were more predictive of hospitalization when concatenated with clinical features (AUC 0.89 vs 0.87). The end-to-end deep learning model had an AUC of 0.80 when initialized using the SSL model trained on all PPG signals, 0.77 when initialized using SSL trained on labelled data only, and 0.73 when initialized randomly. Conclusions: This study shows that SSL can extract features from PPG signals that are predictive of hospitalization or initialize end-to-end deep learning models. Furthermore, SSL can leverage larger unlabelled data sets to improve performance of models fitted using small labelled data sets.

A Multifunctional Network with Uncertainty Estimation and Attention-Based Knowledge Distillation to Address Practical Challenges in Respiration Rate Estimation

Respiration rate is a vital parameter to indicate good health, wellbeing, and performance. As the estimation through classical measurement modes are limited only to rest or during slow movements, respiration rate is commonly estimated through physiological signals such as electrocardiogram and photoplethysmography due to the unobtrusive nature of wearable devices. Deep learning methodologies have gained much traction in the recent past to enhance accuracy during activities involving a lot of movement. However, these methods pose challenges, including model interpretability, uncertainty estimation in the context of respiration rate estimation, and model compactness in terms of deployment in wearable platforms. In this direction, we propose a multifunctional framework, which includes the combination of an attention mechanism, an uncertainty estimation functionality, and a knowledge distillation framework. We evaluated the performance of our framework on two datasets containing ambulatory movement. The attention mechanism visually and quantitatively improved instantaneous respiration rate estimation. Using Monte Carlo dropouts to embed the network with inferential uncertainty estimation resulted in the rejection of 3.7% of windows with high uncertainty, which consequently resulted in an overall reduction of 7.99% in the mean absolute error. The attention-aware knowledge distillation mechanism reduced the model's parameter count and inference time by 49.5% and 38.09%, respectively, without any increase in error rates. Through experimentation, ablation, and visualization, we demonstrated the efficacy of the proposed framework in addressing practical challenges, thus taking a step towards deployment in wearable edge devices.

Dual-Sensor Signals Based Exact Gaussian Process-Assisted Hybrid Feature Extraction and Weighted Feature Fusion for Respiratory Rate and Uncertainty Estimations

Accurately estimating respiratory rate (RR) has become essential for patients and the elderly. Hence, we propose a novel method that uses exact Gaussian process regression (EGPR)-assisted hybrid feature extraction and feature fusion based on photoplethysmography and electrocardiogram signals to improve the reliability of accurate RR and uncertainty estimations. First, we obtain the power spectral features and use the multi-phase feature model to compensate for insufficient input data. Then, we combine four different feature sets and choose features with high weights using a robust neighbor component analysis. The proposed EGPR algorithm provides a confidence interval representing the uncertainty. Therefore, the proposed EGPR algorithm, including hybrid feature extraction and weighted feature fusion, is an excellent model with improved reliability for accurate RR estimation. Furthermore, the proposed EGPR methodology is likely the only one currently available that provides highly stable variation and confidence intervals. The proposed EGPR-MF, 0.993 breath per minute (bpm), and EGPR-feature fusion, 1.064 (bpm), show the lowest mean absolute error compared to the other models.

Breathing Rate Estimation from Head-Worn Photoplethysmography Sensor Data Using Machine Learning

Breathing rate is considered one of the fundamental vital signs and a highly informative indicator of physiological state. Given that the monitoring of heart activity is less complex than the monitoring of breathing, a variety of algorithms have been developed to estimate breathing activity from heart activity. However, estimating breathing rate from heart activity outside of laboratory conditions is still a challenge. The challenge is even greater when new wearable devices with novel sensor placements are being used. In this paper, we present a novel algorithm for breathing rate estimation from photoplethysmography (PPG) data acquired from a head-worn virtual reality mask equipped with a PPG sensor placed on the forehead of a subject. The algorithm is based on advanced signal processing and machine learning techniques and includes a novel quality assessment and motion artifacts removal procedure. The proposed algorithm is evaluated and compared to existing approaches from the related work using two separate datasets that contains data from a total of 37 subjects overall. Numerous experiments show that the proposed algorithm outperforms the compared algorithms, achieving a mean absolute error of 1.38 breaths per minute and a Pearson's correlation coefficient of 0.86. These results indicate that reliable estimation of breathing rate is possible based on PPG data acquired from a head-worn device.

Novel Continuous Respiratory Rate Monitoring Using an Armband Wearable Sensor

Photoplethysmography (PPG) and accelerometer (ACC) are commonly integrated into wearable devices for continuous unobtrusive pulse rate and activity monitoring of individuals during daily life. However, obtaining continuous and clinically accurate respiratory rate measurements using such wearable sensors remains a challenge. This article presents a novel algorithm for estimation of respiration rate (RR) using an upper-arm worn wearable device by deriving multiple respiratory surrogate signals from PPG and ACC sensing. This RR algorithm is retrospectively evaluated on a controlled respiratory clinical testing dataset from 38 subjects with simultaneously recorded wearable sensor data and a standard capnography monitor as an RR reference. The proposed RR method shows great performance and robustness in determining RR measurements over a wide range of 4-59 brpm with an overall bias of -1.3 brpm, mean absolute error (MAE) of 2.7±1.6 brpm, and a meager outage of 0.3±1.2%, while a standard PPG Smart Fusion method produces a bias of -3.6 brpm, an MAE of 5.5±3.1 brpm, and an outage of 0.7±2.5% for direct comparison. In addition, the proposed algorithm showed no significant differences (p=0.63) in accurately determining RR values in subjects with darker skin tones, while the RR performance of the PPG Smart Fusion method is significantly (P<0.001) affected by the darker skin pigmentation. This study demonstrates a highly accurate RR algorithm for unobtrusive continuous RR monitoring using an armband wearable device.

An End-to-End and Accurate PPG-based Respiratory Rate Estimation Approach Using Cycle Generative Adversarial Networks

Respiratory rate (RR) is a clinical sign representing ventilation. An abnormal change in RR is often the first sign of health deterioration as the body attempts to maintain oxygen delivery to its tissues. There has been a growing interest in remotely monitoring of RR in everyday settings which has made photoplethysmography (PPG) monitoring wearable devices an attractive choice. PPG signals are useful sources for RR extraction due to the presence of respiration-induced modulations in them. The existing PPG-based RR estimation methods mainly rely on hand-crafted rules and manual parameters tuning. An end-to-end deep learning approach was recently proposed, however, despite its automatic nature, the performance of this method is not ideal using the real world data. In this paper, we present an end-to-end and accurate pipeline for RR estimation using Cycle Generative Adversarial Networks (CycleGAN) to reconstruct respiratory signals from raw PPG signals. Our results demonstrate a higher RR estimation accuracy of up to 2× (mean absolute error of 1.9±0.3 using five fold cross validation) compared to the state-of-th-art using a identical publicly available dataset. Our results suggest that CycleGAN can be a valuable method for RR estimation from raw PPG signals.

Using self-supervised feature learning to improve the use of pulse oximeter signals to predict paediatric hospitalization

Background: The success of many machine learning applications depends on knowledge about the relationship between the input data and the task of interest (output), hindering the application of machine learning to novel tasks. End-to-end deep learning, which does not require intermediate feature engineering, has been recommended to overcome this challenge but end-to-end deep learning models require large labelled training data sets often unavailable in many medical applications. In this study, we trained machine learning models to predict paediatric hospitalization given raw photoplethysmography (PPG) signals obtained from a pulse oximeter. We trained self-supervised learning (SSL) for automatic feature extraction from PPG signals and assessed the utility of SSL in initializing end-to-end deep learning models trained on a small labelled data set with the aim of predicting paediatric hospitalization.Methods: We compared logistic regression models fitted using features extracted using SSL with end-to-end deep learning models initialized either randomly or using weights from the SSL model. We also compared the performance of SSL models trained on labelled data alone (n=1,031) with SSL trained using both labelled and unlabelled signals (n=7,578). Results: The SSL model trained on both labelled and unlabelled PPG signals produced features that were more predictive of hospitalization compared to the SSL model trained on labelled PPG only (AUC of logistic regression model: 0.78 vs 0.74). The end-to-end deep learning model had an AUC of 0.80 when initialized using the SSL model trained on all PPG signals, 0.77 when initialized using SSL trained on labelled data only, and 0.73 when initialized randomly. Conclusions: This study shows that SSL can improve the classification of PPG signals by either extracting features required by logistic regression models or initializing end-to-end deep learning models. Furthermore, SSL can leverage larger unlabelled data sets to improve performance of models fitted using small labelled data sets.

Measurement of respiratory rate using wearable devices and applications to COVID-19 detection

We show that heart rate enabled wearable devices can be used to measure respiratory rate. Respiration modulates the heart rate creating excess power in the heart rate variability at a frequency equal to the respiratory rate, a phenomenon known as respiratory sinus arrhythmia. We isolate this component from the power spectral density of the heart beat interval time series, and show that the respiratory rate thus estimated is in good agreement with a validation dataset acquired from sleep studies (root mean squared error = 0.648 min^-1, mean absolute error = 0.46 min^-1, mean absolute percentage error = 3%). We use this respiratory rate algorithm to illuminate two potential applications (a) understanding the distribution of nocturnal respiratory rate as a function of age and sex, and (b) examining changes in longitudinal nocturnal respiratory rate due to a respiratory infection such as COVID-19. 90% of respiratory rate values for healthy adults fall within the range 11.8-19.2 min^-1 with a mean value of 15.4 min^-1. Respiratory rate is shown to increase with nocturnal heart rate. It also varies with BMI, reaching a minimum at 25 kg/m², and increasing for lower and higher BMI. The respiratory rate decreases slightly with age and is higher in females compared to males for age <50 years, with no difference between females and males thereafter. The 90% range for the coefficient of variation in a 14 day period for females (males) varies from 2.3-9.2% (2.3-9.5%) for ages 20-24 yr, to 2.5-16.8% (2.7-21.7%) for ages 65-69 yr. We show that respiratory rate is often elevated in subjects diagnosed with COVID-19. In a 7 day window from D_-1 to D₊₅ (where D₀ is the date when symptoms first present, for symptomatic individuals, and the test date for asymptomatic cases), we find that 36.4% (23.7%) of symptomatic (asymptomatic) individuals had at least one measurement of respiratory rate 3 min^-1 higher than the regular rate.

Application of a Machine Learning Algorithms in a Wrist-Wearable Sensor for Patient Health Monitoring during Autonomous Hospital Bed Transport

Smart sensors, coupled with artificial intelligence (AI)-enabled remote automated monitoring (RAMs), can free a nurse from the task of in-person patient monitoring during the transportation process of patients between different wards in hospital settings. Automation of hospital beds using advanced robotics and sensors has been a growing trend exacerbated by the COVID crisis. In this exploratory study, a polynomial regression (PR) machine learning (ML) RAM algorithm based on a Dreyfusian descriptor for immediate wellbeing monitoring was proposed for the autonomous hospital bed transport (AHBT) application. This method was preferred over several other AI algorithm for its simplicity and quick computation. The algorithm quantified historical data using supervised photoplethysmography (PPG) data for 5 min just before the start of the autonomous journey, referred as pre-journey (PJ) dataset. During the transport process, the algorithm continued to quantify immediate measurements using non-overlapping sets of 30 PPG waveforms, referred as in-journey (IJ) dataset. In combination, this algorithm provided a binary decision condition that determined if AHBT should continue its journey to destination by checking the degree of polynomial (DoP) between PJ and IJ. Wrist PPG was used as algorithm’s monitoring parameter. PPG data was collected simultaneously from both wrists of 35 subjects, aged 21 and above in postures mimicking that in AHBT and were given full freedom of upper limb and wrist movement. It was observed that the top goodness-of-fit which indicated potentials for high data accountability had 0.2 to 0.6 cross validation score mean (CVSM) occurring at 8th to 10th DoP for PJ datasets and 0.967 to 0.994 CVSM at 9th to 10th DoP for IJ datasets. CVSM was a reliable metric to pick out the best PJ and IJ DoPs. Central tendency analysis showed that coinciding DoP distributions between PJ and IJ datasets, peaking at 8th DoP, was the precursor to high algorithm stability. Mean algorithm efficacy was 0.20 as our proposed algorithm was able to pick out all signals from a conscious subject having full freedom of movement. This efficacy was acceptable as a first ML proof of concept for AHBT. There was no observable difference between subjects’ left and right wrists.