|
1
|
Sieberts SK, Schaff J, Duda M, Pataki BÁ, Sun M, Snyder P, Daneault JF, Parisi F, Costante G, Rubin U, Banda P, Chae Y, Chaibub Neto E, Dorsey ER, Aydın Z, Chen A, Elo LL, Espino C, Glaab E, Goan E, Golabchi FN, Görmez Y, Jaakkola MK, Jonnagaddala J, Klén R, Li D, McDaniel C, Perrin D, Perumal TM, Rad NM, Rainaldi E, Sapienza S, Schwab P, Shokhirev N, Venäläinen MS, Vergara-Diaz G, Zhang Y, Wang Y, Guan Y, Brunner D, Bonato P, Mangravite LM, Omberg L. Crowdsourcing digital health measures to predict Parkinson's disease severity: the Parkinson's Disease Digital Biomarker DREAM Challenge. NPJ Digit Med 2021; 4:53. [PMID: 33742069 PMCID: PMC7979931 DOI: 10.1038/s41746-021-00414-7] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/16/2020] [Accepted: 02/08/2021] [Indexed: 12/16/2022] Open
Abstract
Consumer wearables and sensors are a rich source of data about patients' daily disease and symptom burden, particularly in the case of movement disorders like Parkinson's disease (PD). However, interpreting these complex data into so-called digital biomarkers requires complicated analytical approaches, and validating these biomarkers requires sufficient data and unbiased evaluation methods. Here we describe the use of crowdsourcing to specifically evaluate and benchmark features derived from accelerometer and gyroscope data in two different datasets to predict the presence of PD and severity of three PD symptoms: tremor, dyskinesia, and bradykinesia. Forty teams from around the world submitted features, and achieved drastically improved predictive performance for PD status (best AUROC = 0.87), as well as tremor- (best AUPR = 0.75), dyskinesia- (best AUPR = 0.48) and bradykinesia-severity (best AUPR = 0.95).
Collapse
Affiliation(s)
| | | | - Marlena Duda
- Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, MI, USA
| | - Bálint Ármin Pataki
- Department of Physics of Complex Systems, ELTE Eötvös Loránd University, Budapest, Hungary
| | | | | | - Jean-Francois Daneault
- Dept of PM&R, Harvard Medical School, Spaulding Rehabilitation Hospital, Charlestown, MA, USA
- Dept of Rehabilitation and Movement Sciences, Rutgers University, Newark, NJ, USA
| | - Federico Parisi
- Dept of PM&R, Harvard Medical School, Spaulding Rehabilitation Hospital, Charlestown, MA, USA
- Wyss Institute, Harvard University, Boston, MA, USA
| | - Gianluca Costante
- Dept of PM&R, Harvard Medical School, Spaulding Rehabilitation Hospital, Charlestown, MA, USA
- Wyss Institute, Harvard University, Boston, MA, USA
| | - Udi Rubin
- Early Signal Foundation, 311 W 43rd Street, New York, NY, USA
| | - Peter Banda
- Luxembourg Centre for Systems Biomedicine, University of Luxembourg, Esch-sur-Alzette, Luxembourg
| | | | | | - E Ray Dorsey
- Center for Health + Technology, University of Rochester, Rochester, NY, USA
| | - Zafer Aydın
- Department of Electrical and Computer Engineering, Abdullah Gul University, Kayseri, Turkey
| | - Aipeng Chen
- Prince of Wales Clinical School, UNSW Sydney, Sydney, Australia
| | - Laura L Elo
- Turku Bioscience Centre, University of Turku and Åbo Akademi University, Tykistökatu 6, Turku, Finland
| | - Carlos Espino
- Early Signal Foundation, 311 W 43rd Street, New York, NY, USA
| | - Enrico Glaab
- Luxembourg Centre for Systems Biomedicine, University of Luxembourg, Esch-sur-Alzette, Luxembourg
| | - Ethan Goan
- School of Electrical Engineering and Robotics, Queensland University of Technology, Brisbane, QLD, Australia
| | | | - Yasin Görmez
- Department of Electrical and Computer Engineering, Abdullah Gul University, Kayseri, Turkey
| | - Maria K Jaakkola
- Turku Bioscience Centre, University of Turku and Åbo Akademi University, Tykistökatu 6, Turku, Finland
- Department of Mathematics and Statistics, University of Turku, Turku, Finland
| | - Jitendra Jonnagaddala
- School of Public Health and Community Medicine, UNSW Sydney, Sydney, Australia
- WHO Collaborating Centre for eHealth, UNSW Sydney, Sydney, Australia
| | - Riku Klén
- Turku Bioscience Centre, University of Turku and Åbo Akademi University, Tykistökatu 6, Turku, Finland
| | - Dongmei Li
- Clinical and Translational Science Institute, University of Rochester Medical Center, Rochester, NY, USA
| | - Christian McDaniel
- Artificial Intelligence, University of Georgia, Athens, GA, USA
- Computer Science, University of Georgia, Athens, GA, USA
| | - Dimitri Perrin
- School of Computer Science, Queensland University of Technology, Brisbane, QLD, Australia
| | | | - Nastaran Mohammadian Rad
- Institute for Computing and Information Sciences, Radboud University, Nijmegen, The Netherlands
- Fondazione Bruno Kessler (FBK), Via Sommarive 18, Povo, Trento, Italy
- University of Trento, Trento, Italy
| | - Erin Rainaldi
- Verily Life Sciences, 269 East Grand Ave, South San Francisco, CA, USA
| | - Stefano Sapienza
- Dept of PM&R, Harvard Medical School, Spaulding Rehabilitation Hospital, Charlestown, MA, USA
| | - Patrick Schwab
- Institute of Robotics and Intelligent Systems, ETH Zurich, Zurich, Switzerland
| | | | - Mikko S Venäläinen
- Turku Bioscience Centre, University of Turku and Åbo Akademi University, Tykistökatu 6, Turku, Finland
| | - Gloria Vergara-Diaz
- Dept of PM&R, Harvard Medical School, Spaulding Rehabilitation Hospital, Charlestown, MA, USA
| | - Yuqian Zhang
- School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - Yuanjia Wang
- Department of Biostatistics, Mailman School of Public Health, Columbia University, New York, NY, USA
| | - Yuanfang Guan
- Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, MI, USA
| | - Daniela Brunner
- Early Signal Foundation, 311 W 43rd Street, New York, NY, USA
- Dept. of Psychiatry, Columbia University, New York, NY, USA
| | - Paolo Bonato
- Dept of PM&R, Harvard Medical School, Spaulding Rehabilitation Hospital, Charlestown, MA, USA
- Wyss Institute, Harvard University, Boston, MA, USA
| | | | | |
Collapse
|