1
|
Pires GN, Arnardóttir ES, Bailly S, McNicholas WT. Guidelines for the development, performance evaluation and validation of new sleep technologies (DEVSleepTech guidelines) - a protocol for a Delphi consensus study. J Sleep Res 2024; 33:e14163. [PMID: 38351277 DOI: 10.1111/jsr.14163] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/16/2023] [Revised: 01/22/2024] [Accepted: 01/23/2024] [Indexed: 10/18/2024]
Abstract
New sleep technologies are being developed, refined and delivered at a fast pace. However, there are serious concerns about the validation and accuracy of new sleep-related technologies being made available, as many of them, especially consumer-sleep technologies, have not been tested in comparison with gold-standard methods or have been approved by health regulatory agencies. The importance of proper validation and performance evaluation of new sleep technologies has already been discussed in previous studies and some recommendations have already been published, but most of them do not employ standardized methodology and are not able to cover all aspects of new sleep technologies. The current protocol describes the methods of a Delphi consensus study to create guidelines for the development, performance evaluation and validation of new sleep devices and technologies. The resulting recommendations are not intended to be used as a quality assessment tool to evaluate individual articles, but rather to evaluate the overall procedures, studies and experiments performed to develop, evaluate performance and validate new technologies. We hope these guidelines can be helpful for researchers who work with new sleep technologies on the appraisal of their reliability and validation, for companies who are working on the development and refinement of new sleep technologies, and by regulatory agencies to evaluate new technologies that are looking for registration, approval or inclusion on health systems.
Collapse
Affiliation(s)
- Gabriel Natan Pires
- Departamento de Psicobiologia, Universidade Federal de São Paulo, São Paulo, Brazil
- Sleep Institute, São Paulo, Brazil
- European Sleep Research Society (ESRS), Regensburg, Germany
| | - Erna S Arnardóttir
- Reykjavik University Sleep Institute, Reykjavik University, Reykjavik, Iceland
- Landspitali, The National University Hospital of Iceland, Reykjavik, Iceland
| | - Sébastien Bailly
- Grenoble Alpes University, Inserm U1300, Grenoble Alpes University Hospital, Grenoble, France
| | - Walter T McNicholas
- School of Medicine and the Conway Research Institute, University College Dublin, Dublin, Ireland
- Department of Respiratory and Sleep Medicine, St Vincent's Hospital Group, Dublin, Ireland
| |
Collapse
|
2
|
Landry V, Semsar-Kazerooni K, Chen T, Gurberg J, Nguyen LHP, Constantin E. Diagnostic accuracy of portable sleep monitors in pediatric sleep apnea: A systematic review. Sleep Med Rev 2024; 78:101991. [PMID: 39173472 DOI: 10.1016/j.smrv.2024.101991] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/11/2023] [Revised: 05/01/2024] [Accepted: 08/05/2024] [Indexed: 08/24/2024]
Abstract
In recent years, a plethora of new type III and IV portable sleep monitors (PSM) have been developed, although evidence regarding their diagnostic accuracy for use in children remains heterogeneous. This study systematically reviews the literature addressing the diagnostic accuracies of type III and IV PSM for pediatric sleep apnea. Publications indexed in Medline, Embase, or Web of Science were reviewed using the PRISMA framework. Of 1054 studies, 62 fulfilled the inclusion criteria. Of the studies evaluating oximetry-based type IV PSM, one (6.25 %) demonstrated a balanced set of high (≥80 %) sensitivities and specificities for the diagnosis of any pediatric sleep apnea, while five studies (27.8 %) showed similar accuracies for moderate-to-severe sleep apnea. For non-oximetry-based type IV PSM, two studies (40 %) reported a balanced set of high diagnostic accuracies for moderate-to-severe sleep apnea. Type III PSM repeatedly demonstrated higher diagnostic accuracies, with six studies (66.7 %) reporting a balanced set of high diagnostic accuracies for moderate-to-severe sleep apnea. This review highlights the potential of type III PSM to detect moderate-to-severe pediatric sleep apnea, although current evidence is limited to support the stand-alone use of type IV PSM for the diagnosis of sleep apnea in most children.
Collapse
Affiliation(s)
- Vivianne Landry
- Division of Otolaryngology-Head and Neck Surgery, University of Montreal, Montreal, QC, Canada
| | | | - Tanya Chen
- Department of Otolaryngology-Head and Neck Surgery, University of Toronto, Toronto, ON, Canada
| | - Joshua Gurberg
- Department of Otolaryngology-Head and Neck Surgery, McGill University, Montreal, QC, Canada
| | - Lily H P Nguyen
- Department of Otolaryngology-Head and Neck Surgery, McGill University, Montreal, QC, Canada
| | - Evelyn Constantin
- Department of Pediatrics, Pediatric Sleep Medicine, McGill University, Montreal, QC, Canada.
| |
Collapse
|
3
|
Zhong Y, Jatav A, Afrin K, Shivaram T, Bukkapatnam STS. Enhanced SpO 2 estimation using explainable machine learning and neck photoplethysmography. Artif Intell Med 2023; 145:102685. [PMID: 37925216 DOI: 10.1016/j.artmed.2023.102685] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/25/2022] [Revised: 10/10/2023] [Accepted: 10/11/2023] [Indexed: 11/06/2023]
Abstract
Reflectance-based photoplethysmogram (PPG) sensors provide flexible options of measuring sites for blood oxygen saturation (SpO2) measurement. But they are mostly limited by accuracy, especially when applied to different subjects, due to the diverse human characteristics (skin colors, hair density, etc.) and usage conditions of different sensor settings. This study addresses the estimation of SpO2 at non-standard measuring sites employing reflectance-based sensors. It proposes an automated construction of subject inclusion-exclusion criteria for SpO2 measuring devices, using a combination of unsupervised clustering, supervised regression, and model explanations. This is perhaps among the first adaptation of SHAP to explain the clusters gleaned from unsupervised learning methods. As a wellness application case study, we developed a pillow-based wearable device to collect reflectance PPGs from both the brachiocephalic and carotid arteries around the neck. The experiment was conducted on 33 subjects, each under totally 80 different sensor settings. The proposed approach addressed the variations of humans and devices, as well as the heterogeneous mapping between signals and SpO2 values. It identified effective device settings and characteristics of their applicable subject groups (i.e., subject inclusion-exclusion criteria). Overall, it reduced the root mean squared error (RMSE) by 16%, compared to an empirical formula and a plain SpO2 estimation model.
Collapse
Affiliation(s)
- Yuhao Zhong
- Wm Michael Barnes '64 Department of Industrial & Systems Engineering, Texas A&M University, College Station, TX 77840, USA.
| | - Ashish Jatav
- Wm Michael Barnes '64 Department of Industrial & Systems Engineering, Texas A&M University, College Station, TX 77840, USA.
| | - Kahkashan Afrin
- Wm Michael Barnes '64 Department of Industrial & Systems Engineering, Texas A&M University, College Station, TX 77840, USA.
| | - Tejaswini Shivaram
- Department of Electrical & Computer Engineering, Texas A&M University, College Station, TX 77840, USA.
| | - Satish T S Bukkapatnam
- Wm Michael Barnes '64 Department of Industrial & Systems Engineering, Texas A&M University, College Station, TX 77840, USA.
| |
Collapse
|
4
|
Pires GN, Arnardóttir ES, Islind AS, Leppänen T, McNicholas WT. Consumer sleep technology for the screening of obstructive sleep apnea and snoring: current status and a protocol for a systematic review and meta-analysis of diagnostic test accuracy. J Sleep Res 2023:e13819. [PMID: 36807680 DOI: 10.1111/jsr.13819] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2022] [Revised: 12/16/2022] [Accepted: 12/18/2022] [Indexed: 02/20/2023]
Abstract
There are concerns about the validation and accuracy of currently available consumer sleep technology for sleep-disordered breathing. The present report provides a background review of existing consumer sleep technologies and discloses the methods and procedures for a systematic review and meta-analysis of diagnostic test accuracy of these devices and apps for the detection of obstructive sleep apnea and snoring in comparison with polysomnography. The search will be performed in four databases (PubMed, Scopus, Web of Science, and the Cochrane Library). Studies will be selected in two steps, first by an analysis of abstracts followed by full-text analysis, and two independent reviewers will perform both phases. Primary outcomes include apnea-hypopnea index, respiratory disturbance index, respiratory event index, oxygen desaturation index, and snoring duration for both index and reference tests, as well as the number of true positives, false positives, true negatives, and false negatives for each threshold, as well as for epoch-by-epoch and event-by-event results, which will be considered for the calculation of surrogate measures (including sensitivity, specificity, and accuracy). Diagnostic test accuracy meta-analyses will be performed using the Chu and Cole bivariate binomial model. Mean difference meta-analysis will be performed for continuous outcomes using the DerSimonian and Laird random-effects model. Analyses will be performed independently for each outcome. Subgroup and sensitivity analyses will evaluate the effects of the types (wearables, nearables, bed sensors, smartphone applications), technologies (e.g., oximeter, microphone, arterial tonometry, accelerometer), the role of manufacturers, and the representativeness of the samples.
Collapse
Affiliation(s)
- Gabriel Natan Pires
- Departamento de Psicobiologia, Universidade Federal de São Paulo, São Paulo, Brazil.,European Sleep Research Society (ESRS), Regensburg, Germany
| | - Erna Sif Arnardóttir
- Reykjavik University Sleep Institute, Reykjavik University, Reykjavik, Iceland.,Landspitali-The National University Hospital of Iceland, Reykjavik, Iceland
| | - Anna Sigridur Islind
- Reykjavik University Sleep Institute, Reykjavik University, Reykjavik, Iceland.,Department of Computer Science, Reykjavik University, Reykjavik, Iceland
| | - Timo Leppänen
- Department of Technical Physics, University of Eastern Finland, Kuopio, Finland.,Diagnostic Imaging Center, Kuopio University Hospital, Kuopio, Finland.,School of Information Technology and Electrical Engineering, The University of Queensland, Brisbane, Australia
| | - Walter T McNicholas
- Department of Respiratory and Sleep Medicine, St Vincent's Hospital Group, School of Medicine, University College Dublin, Dublin, Ireland
| |
Collapse
|
5
|
Riha RL, Celmina M, Cooper B, Hamutcu-Ersu R, Kaditis A, Morley A, Pataka A, Penzel T, Roberti L, Ruehland W, Testelmans D, van Eyck A, Grundström G, Verbraecken J, Randerath W. ERS technical standards for using type III devices (limited channel studies) in the diagnosis of sleep disordered breathing in adults and children. Eur Respir J 2023; 61:13993003.00422-2022. [PMID: 36609518 DOI: 10.1183/13993003.00422-2022] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/25/2022] [Accepted: 07/27/2022] [Indexed: 02/01/2023]
Abstract
For more than three decades, type III devices have been used in the diagnosis of sleep disordered breathing in supervised as well as unsupervised settings. They have satisfactory positive and negative predictive values for detecting obstructive and central sleep apnoea in populations with moderately high pre-test probability of symptoms associated with these events. However, standardisation of commercially available type III devices has never been undertaken and the technical specifications can vary widely. None have been subjected to the same rigorous processes as most other diagnostic modalities in the medical field. Although type III devices do not include acquisition of electroencephalographic signals overnight, the minimum number of physical sensors required to allow for respiratory event scoring using standards outlined by the American Academy of Sleep Medicine remains debatable. This technical standard summarises data on type III studies published since 2007 from multiple perspectives in both adult and paediatric sleep practice. Most importantly, it aims to provide a framework for considering current type III device limitations in the diagnosis of sleep disordered breathing while raising research- and practice-related questions aimed at improving our use of these devices in the present and future.
Collapse
Affiliation(s)
- Renata L Riha
- Department of Sleep Medicine, The Royal Infirmary Edinburgh, Edinburgh, UK
| | - Marta Celmina
- Epilepsy and Sleep Medicine Centre, Children's Clinical University Hospital, Riga, Latvia
| | - Brendan Cooper
- Lung Function and Sleep, University Hospitals Birmingham NHS Foundation Trust, Queen Elizabeth Hospital Birmingham, Edgbaston, UK
| | | | - Athanasios Kaditis
- Division of Paediatric Pulmonology and Sleep Disorders Laboratory, First Department of Pediatrics, National and Kapodistrian University of Athens School of Medicine and Agia Sofia Children's Hospital, Athens, Greece
| | | | - Athanasia Pataka
- Respiratory Failure Unit, G. Papanikolaou Hospital, Aristotle University of Thessaloniki, Thessaloniki, Greece
| | - Thomas Penzel
- Department of Cardiology and Angiology, Interdisciplinary Center of Sleep Medicine, Charité Universitätsmedizin Berlin, Berlin, Germany
| | | | - Warren Ruehland
- Institute for Breathing and Sleep, Austin Health, Melbourne, Australia
| | - Dries Testelmans
- Department of Pneumology, University Hospitals Leuven, Leuven, Belgium
| | - Annelies van Eyck
- Laboratory of Experimental Medicine and Pediatrics, University of Antwerp, Antwerp (Edegem), Belgium
- Department of Pediatrics, Antwerp University Hospital, Antwerp (Edegem), Belgium
| | | | - Johan Verbraecken
- Antwerp University Hospital and University of Antwerp, Edegem (Antwerp), Belgium
| | - Winfried Randerath
- Bethanien Hospital, Clinic of Pneumology and Allergology, Center for Sleep Medicine and Respiratory Care, Institute of Pneumology at the University of Cologne, Solingen, Germany
| |
Collapse
|
6
|
Santilli M, Manciocchi E, D’Addazio G, Di Maria E, D’Attilio M, Femminella B, Sinjari B. Prevalence of Obstructive Sleep Apnea Syndrome: A Single-Center Retrospective Study. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2021; 18:ijerph181910277. [PMID: 34639577 PMCID: PMC8508429 DOI: 10.3390/ijerph181910277] [Citation(s) in RCA: 22] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/05/2021] [Revised: 09/13/2021] [Accepted: 09/26/2021] [Indexed: 12/15/2022]
Abstract
Obstructive sleep apnea syndrome (OSAS) is a sleep breathing disorder that often remains undiagnosed and untreated. OSAS prevalence is increasing exponentially. Starting on the dentist's role as an epidemiological and diagnostic "sentinel", the purpose of this study was to assess the prevalence of OSAS. The clinical diaries of 4659 patients were reviewed through a single-center retrospective analytic study. Descriptive statistical analysis was performed. Only 0.26% of patients reported to suffer from sleep apnea and were then diagnosed with OSAS. It was found that, out of 4487 patients, 678 suffered from hypertension (14.80%), 188 from gastro-esophageal-reflux-disease (GERD = 4.10%) and 484 from gastritis (10.78%). These results could be related to a difficult diagnosis of OSAS and to the absence of a dedicated section on sleep disorders in medical records. Therefore, the introduction of a question dedicated to sleep disorders, the administration of questionnaires (such as the STOP-BANG questionnaire) for early diagnosis, a multidisciplinary approach and pneumological examination could support the dentist in identifying patients at risk of OSAS.
Collapse
Affiliation(s)
- Manlio Santilli
- Unit of Prosthodontics, Department of Innovative Technologies in Medicine and Dentistry, University “G. d’Annunzio” Chieti-Pescara, 66100 Chieti, Italy; (M.S.); (E.M.); (G.D.); (B.F.)
| | - Eugenio Manciocchi
- Unit of Prosthodontics, Department of Innovative Technologies in Medicine and Dentistry, University “G. d’Annunzio” Chieti-Pescara, 66100 Chieti, Italy; (M.S.); (E.M.); (G.D.); (B.F.)
| | - Gianmaria D’Addazio
- Unit of Prosthodontics, Department of Innovative Technologies in Medicine and Dentistry, University “G. d’Annunzio” Chieti-Pescara, 66100 Chieti, Italy; (M.S.); (E.M.); (G.D.); (B.F.)
| | - Erica Di Maria
- Unit of Orthodontics, Department of Innovative Technologies in Medicine and Dentistry, University “G. d’Annunzio” Chieti-Pescara, 66100 Chieti, Italy; (E.D.M.); (M.D.)
| | - Michele D’Attilio
- Unit of Orthodontics, Department of Innovative Technologies in Medicine and Dentistry, University “G. d’Annunzio” Chieti-Pescara, 66100 Chieti, Italy; (E.D.M.); (M.D.)
| | - Beatrice Femminella
- Unit of Prosthodontics, Department of Innovative Technologies in Medicine and Dentistry, University “G. d’Annunzio” Chieti-Pescara, 66100 Chieti, Italy; (M.S.); (E.M.); (G.D.); (B.F.)
| | - Bruna Sinjari
- Unit of Prosthodontics, Department of Innovative Technologies in Medicine and Dentistry, University “G. d’Annunzio” Chieti-Pescara, 66100 Chieti, Italy; (M.S.); (E.M.); (G.D.); (B.F.)
- Correspondence: ; Tel.: +39-392-27471479; Fax: +39-0871-3554070
| |
Collapse
|
7
|
Automatic sleep stage classification with reduced epoch of EEG. EVOLUTIONARY INTELLIGENCE 2021. [DOI: 10.1007/s12065-021-00632-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
|