Performance evaluation of a wrist-worn reflectance pulse oximeter during sleep

Performance of a commercial smart watch compared to polysomnography reference for overnight continuous oximetry measurement and sleep apnea evaluation

STUDY OBJECTIVES

We evaluated the accuracy and precision of continuous overnight oxygen saturation (SpO2) measurement by a commercial wrist device (WD) incorporating high-grade sensors and investigated WD estimation of sleep-disordered breathing by quantifying overnight oxygen desaturation index compared to polysomnography (PSG) oxygen desaturation index and apnea-hypopnea index (AHI) with and without sleep questionnaire data to assess the WD's ability to detect obstructive sleep apnea and determine its severity.

METHODS

Participants completed sleep questionnaires, had a WD (Samsung Galaxy Watch 4) placed on their wrist, and underwent attended, in-laboratory overnight PSG (Nihon Kohden) with a pulse oximetry probe secured either to a finger or an ear lobe. PSG data were scored by a single experienced registered PSG technologist. Statistical analysis included demographic characteristics, continuous SpO2 measurement WD vs PSG root-mean-square error with Bland-Altman plot and linear regression associations. Predictive models for PSG oxygen desaturation index and AHI severity were built using logistic regression with probability cutoffs determined via receiver operating curve characteristics.

RESULTS

The 51 participants analyzed had a median age of 49 (range, 22-78) years; 66.7% were male, with median body mass index of 28.1 (range, 20.1-47.3) kg/m2 with a race/ethnicity distribution of 49.0% Caucasian, 25.5% Hispanic, 9.8% African American, 9.8% Asian, and 5.9% Middle Eastern. WD vs PSG continuous SpO2 measurement in percentage points demonstrated a bias of 0.91 (95% confidence interval, 0.38, 1.45), standard deviation of 2.37 (95% confidence interval, 2.36, 2.38), and root-mean-square error of 2.54 (95% confidence interval, 2.34, 2.73). WD area under the curve receiver operating curve characteristics for predicting PSG were 0.882 oxygen desaturation index > 15 events/h, 0.894 AHI > 30 events/h, 0.800 AHI > 15 events/h, and 0.803 AHI > 5 events/h. WD plus select sleep questionnaire areas under the curve for predicting PSG were 0.943 AHI > 30 events/h, 0.868 AHI > 15 events/h, and 0.863 AHI > 5 events/h.

CONCLUSIONS

The WD conducted reliable overnight continuous SpO2 monitoring with root-mean-square error < 3% vs PSG. Predictive models of PSG AHI based on WD measurements alone, or plus sleep questionnaires, demonstrated excellent to outstanding discrimination for obstructive sleep apnea identification and severity. Longitudinal WD use should be evaluated promptly based on the WD's potential to improve accessibility and accuracy of obstructive sleep apnea testing, as well as support treatment follow-up.

CITATION

Browne SH, Vaida F, Umlauf A, Kim J, DeYoung P, Owens RL. Performance of a commercial smart watch compared to polysomnography reference for overnight continuous oximetry measurement and sleep apnea evaluation. J Clin Sleep Med. 2024;20(9):1479-1488.

Sleep staging algorithm based on smartwatch sensors for healthy and sleep apnea populations

OBJECTIVE

Sleep stages can provide valuable insights into an individual's sleep quality. By leveraging movement and heart rate data collected by modern smartwatches, it is possible to enable the sleep staging feature and enhance users' understanding about their sleep and health conditions.

METHOD

In this paper, we present and validate a recurrent neural network based model with 23 input features extracted from accelerometer and photoplethysmography sensors data for both healthy and sleep apnea populations. We designed a lightweight and fast solution to enable the prediction of sleep stages for each 30-s epoch. This solution was developed using a large dataset of 1522 night recordings collected from a highly heterogeneous population and different versions of Samsung smartwatch.

RESULTS

In the classification of four sleep stages (wake, light, deep, and rapid eye movements sleep), the proposed solution achieved 71.6 % of balanced accuracy and a Cohen's kappa of 0.56 in a test set with 586 recordings.

CONCLUSION

The results presented in this paper validate our proposal as a competitive wearable solution for sleep staging. Additionally, the use of a large and diverse data set contributes to the robustness of our solution, and corroborates the validation of algorithm's performance. Some additional analysis performed for healthy and sleep apnea population demonstrated that algorithm's performance has low correlation with demographic variables.

The 2023 wearable photoplethysmography roadmap

Photoplethysmography is a key sensing technology which is used in wearable devices such as smartwatches and fitness trackers. Currently, photoplethysmography sensors are used to monitor physiological parameters including heart rate and heart rhythm, and to track activities like sleep and exercise. Yet, wearable photoplethysmography has potential to provide much more information on health and wellbeing, which could inform clinical decision making. This Roadmap outlines directions for research and development to realise the full potential of wearable photoplethysmography. Experts discuss key topics within the areas of sensor design, signal processing, clinical applications, and research directions. Their perspectives provide valuable guidance to researchers developing wearable photoplethysmography technology.

Validating a Consumer Smartwatch for Nocturnal Respiratory Rate Measurements in Sleep Monitoring

Wrist-based respiratory rate (RR) measurement during sleep faces accuracy limitations. This study aimed to assess the accuracy of the RR estimation function during sleep based on the severity of obstructive sleep apnea (OSA) using the Samsung Galaxy Watch (GW) series. These watches are equipped with accelerometers and photoplethysmography sensors for RR estimation. A total of 195 participants visiting our sleep clinic underwent overnight polysomnography while wearing the GW, and the RR estimated by the GW was compared with the reference RR obtained from the nasal thermocouple. For all participants, the root mean squared error (RMSE) of the average overnight RR and continuous RR measurements were 1.13 bpm and 1.62 bpm, respectively, showing a small bias of 0.39 bpm and 0.37 bpm, respectively. The Bland-Altman plots indicated good agreement in the RR measurements for the normal, mild, and moderate OSA groups. In participants with normal-to-moderate OSA, both average overnight RR and continuous RR measurements achieved accuracy rates exceeding 90%. However, for patients with severe OSA, these accuracy rates decreased to 79.45% and 75.8%, respectively. The study demonstrates the GW's ability to accurately estimate RR during sleep, even though accuracy may be compromised in patients with severe OSA.

Digital Platform for Continuous Monitoring of Patients Using a Smartwatch: Longitudinal Prospective Cohort Study

BACKGROUND

Since the COVID-19 pandemic, there has been a boost in the digital transformation of the human society, where wearable devices such as a smartwatch can already measure vital signs in a continuous and naturalistic way; however, the security and privacy of personal data is a challenge to expanding the use of these data by health professionals in clinical follow-up for decision-making. Similar to the European General Data Protection Regulation, in Brazil, the Lei Geral de Proteção de Dados established rules and guidelines for the processing of personal data, including those used for patient care, such as those captured by smartwatches. Thus, in any telemonitoring scenario, there is a need to comply with rules and regulations, making this issue a challenge to overcome.

OBJECTIVE

This study aimed to build a digital solution model for capturing data from wearable devices and making them available in a safe and agile manner for clinical and research use, following current laws.

METHODS

A functional model was built following the Brazilian Lei Geral de Proteção de Dados (2018), where data captured by smartwatches can be transmitted anonymously over the Internet of Things and be identified later within the hospital. A total of 80 volunteers were selected for a 24-week follow-up clinical trial divided into 2 groups, one group with a previous diagnosis of COVID-19 and a control group without a previous diagnosis of COVID-19, to measure the synchronization rate of the platform with the devices and the accuracy and precision of the smartwatch in out-of-hospital conditions to simulate remote monitoring at home.

RESULTS

In a 35-week clinical trial, >11.2 million records were collected with no system downtime; 66% of continuous beats per minute were synchronized within 24 hours (79% within 2 days and 91% within a week). In the limit of agreement analysis, the mean differences in oxygen saturation, diastolic blood pressure, systolic blood pressure, and heart rate were -1.280% (SD 5.679%), -1.399 (SD 19.112) mm Hg, -1.536 (SD 24.244) mm Hg, and 0.566 (SD 3.114) beats per minute, respectively. Furthermore, there was no difference in the 2 study groups in terms of data analysis (neither using the smartwatch nor the gold-standard devices), but it is worth mentioning that all volunteers in the COVID-19 group were already cured of the infection and were highly functional in their daily work life.

CONCLUSIONS

On the basis of the results obtained, considering the validation conditions of accuracy and precision and simulating an extrahospital use environment, the functional model built in this study is capable of capturing data from the smartwatch and anonymously providing it to health care services, where they can be treated according to the legislation and be used to support clinical decisions during remote monitoring.

Consumer Wearable Health and Fitness Technology in Cardiovascular Medicine: JACC State-of-the-Art Review

The use of consumer wearable devices (CWDs) to track health and fitness has rapidly expanded over recent years because of advances in technology. The general population now has the capability to continuously track vital signs, exercise output, and advanced health metrics. Although understanding of basic health metrics may be intuitive (eg, peak heart rate), more complex metrics are derived from proprietary algorithms, differ among device manufacturers, and may not historically be common in clinical practice (eg, peak V˙O2, exercise recovery scores). With the massive expansion of data collected at an individual patient level, careful interpretation is imperative. In this review, we critically analyze common health metrics provided by CWDs, describe common pitfalls in CWD interpretation, provide recommendations for the interpretation of abnormal results, present the utility of CWDs in exercise prescription, examine health disparities and inequities in CWD use and development, and present future directions for research and development.

Consumer sleep technology for the screening of obstructive sleep apnea and snoring: current status and a protocol for a systematic review and meta-analysis of diagnostic test accuracy

There are concerns about the validation and accuracy of currently available consumer sleep technology for sleep-disordered breathing. The present report provides a background review of existing consumer sleep technologies and discloses the methods and procedures for a systematic review and meta-analysis of diagnostic test accuracy of these devices and apps for the detection of obstructive sleep apnea and snoring in comparison with polysomnography. The search will be performed in four databases (PubMed, Scopus, Web of Science, and the Cochrane Library). Studies will be selected in two steps, first by an analysis of abstracts followed by full-text analysis, and two independent reviewers will perform both phases. Primary outcomes include apnea-hypopnea index, respiratory disturbance index, respiratory event index, oxygen desaturation index, and snoring duration for both index and reference tests, as well as the number of true positives, false positives, true negatives, and false negatives for each threshold, as well as for epoch-by-epoch and event-by-event results, which will be considered for the calculation of surrogate measures (including sensitivity, specificity, and accuracy). Diagnostic test accuracy meta-analyses will be performed using the Chu and Cole bivariate binomial model. Mean difference meta-analysis will be performed for continuous outcomes using the DerSimonian and Laird random-effects model. Analyses will be performed independently for each outcome. Subgroup and sensitivity analyses will evaluate the effects of the types (wearables, nearables, bed sensors, smartphone applications), technologies (e.g., oximeter, microphone, arterial tonometry, accelerometer), the role of manufacturers, and the representativeness of the samples.

Diagnostic Performance of Photoplethysmography-Based Smartwatch for Obstructive Sleep Apnea

Background and Objectives

Considering the prevalence and health effects of obstructive sleep apnea (OSA), early diagnosis and proper treatment are essential. Polysomnography (PSG) has limitations in diagnosing or tracking large-scale OSA patients. Smartwatches (SWs) can be equipped with a photoplethysmograph (PPG) that can indirectly measure heart rate and blood oxygen saturation by detecting the difference of light absorption through blood. The purpose of this study is to compare oxygen saturation parameters of PPG-based SWs with those of PSG to determine the diagnostic accuracy for OSA.

Methods

After obtaining voluntary consent from patients who were scheduled to undergo PSG in a sleep clinic due to suspected OSA, they were randomly assigned to wear a Galaxy watch4 (GW) or Apple watch7 (AW) on their wrist. The agreement rates between the oxygen saturation parameters of the two SW types and PSG were evaluated. The accuracy, sensitivity, and specificity of the oxygen saturation parameters for diagnosis of OSA (apnea-hypopnea index [AHI] ≥5/h) were compared between the two types of SW.

Results

A total of 133 patients underwent PSG while wearing an SW. Including duplicates, 109 patients wearing a GW and 69 wearing an AW were included. The diagnostic accuracy of AHI ≥5/h according to oxygen saturation time measured by a GW was less than 90%, the respective sensitivity and specificity were 82.9% and 75.8%. The area under the curve (AUC) of the receiver operating characteristic (ROC) curve was 0.807 (p<0.001). Using the lowest oxygen saturation value of GW, the sensitivity was 81.6%, the specificity was 69.7%, and the AUC of the ROC was 0.849 (p<0.001). The diagnostic accuracy of AHI ≥5/h according to the average oxygen saturation of AW, and the sensitivity and specificity were 75.6% and 70.8%, respectively. The AUC of this ROC curve was 0.757 (p<0.001). Using the lowest oxygen saturation value of AW, the sensitivity was 71.1%, the specificity was 62.5%, and the AUC of the ROC was 0.705 (p=0.005).

Conclusion

This study found that the two types of SW showed considerable accuracy in diagnosing OSA, but the accuracy decreased as the severity of OSA increased.

Digital Platform to Continuously Monitor Patients Using a Smartwatch: Preliminary Report

Background

Monitoring vital signs such as oximetry, blood pressure, and heart rate is important to follow the evolution of patients. Smartwatches are a revolution in medicine allowing the collection of such data in a continuous and organic way. However, it is still a challenge to make this information available to health care professionals to make decisions during clinical follow-up.

Objective

This study aims to build a digital solution that displays vital sign data from smartwatches, collected remotely, continuously, reliably, and from multiple users, with trigger warnings when abnormal results are identified.

Methods

This is a single-center prospective study following the guidelines “Evaluating digital health products” from the UK Health Security Agency. A digital platform with 3 different applications was created to capture and display data from the mobile phones of volunteers with smartwatches. We selected 80 volunteers who were followed for 24 weeks each, and the synchronization interval between the smartwatch and digital solution was recorded for each vital sign collected.

Results

In 14 weeks of project progress, we managed to recruit 80 volunteers, with 68 already registered in the digital solution. More than 2.8 million records have already been collected, without system downtime. Less than 5% of continuous heart rate measurements (bpm) were synchronized within 2 hours. However, approximately 70% were synchronized in less than 24 hours, and 90% were synchronized in less than 119 hours.

Conclusions

The digital solution is working properly in its role of displaying data collected from smartwatches. Vital sign values are being monitored by the research team as part of the monitoring of volunteers. Although the digital solution proved unsuitable for monitoring urgent events, it is more than suitable for use in outpatient clinical use. This digital solution, which is based on cloud technology, can be applied in the future for telemonitoring in regions lacking health care professionals. Accuracy and reliability studies still need to be performed at the end of the 24-week follow-up.