1
|
Shuqair M, Jimenez-Shahed J, Ghoraani B. Multi-Shared-Task Self-Supervised CNN-LSTM for Monitoring Free-Body Movement UPDRS-III Using Wearable Sensors. Bioengineering (Basel) 2024; 11:689. [PMID: 39061771 PMCID: PMC11274108 DOI: 10.3390/bioengineering11070689] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2024] [Revised: 06/28/2024] [Accepted: 07/03/2024] [Indexed: 07/28/2024] Open
Abstract
The Unified Parkinson's Disease Rating Scale (UPDRS) is used to recognize patients with Parkinson's disease (PD) and rate its severity. The rating is crucial for disease progression monitoring and treatment adjustment. This study aims to advance the capabilities of PD management by developing an innovative framework that integrates deep learning with wearable sensor technology to enhance the precision of UPDRS assessments. We introduce a series of deep learning models to estimate UPDRS Part III scores, utilizing motion data from wearable sensors. Our approach leverages a novel Multi-shared-task Self-supervised Convolutional Neural Network-Long Short-Term Memory (CNN-LSTM) framework that processes raw gyroscope signals and their spectrogram representations. This technique aims to refine the estimation accuracy of PD severity during naturalistic human activities. Utilizing 526 min of data from 24 PD patients engaged in everyday activities, our methodology demonstrates a strong correlation of 0.89 between estimated and clinically assessed UPDRS-III scores. This model outperforms the benchmark set by single and multichannel CNN, LSTM, and CNN-LSTM models and establishes a new standard in UPDRS-III score estimation for free-body movements compared to recent state-of-the-art methods. These results signify a substantial step forward in bioengineering applications for PD monitoring, providing a robust framework for reliable and continuous assessment of PD symptoms in daily living settings.
Collapse
Affiliation(s)
- Mustafa Shuqair
- Department of Electrical Engineering and Computer Science, Florida Atlantic University, Boca Raton, FL 33431, USA;
| | - Joohi Jimenez-Shahed
- Department of Neurology, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA;
| | - Behnaz Ghoraani
- Department of Electrical Engineering and Computer Science, Florida Atlantic University, Boca Raton, FL 33431, USA;
| |
Collapse
|
2
|
Bibbo D, De Marchis C, Schmid M, Ranaldi S. Machine learning to detect, stage and classify diseases and their symptoms based on inertial sensor data: a mapping review. Physiol Meas 2023; 44:12TR01. [PMID: 38061062 DOI: 10.1088/1361-6579/ad133b] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/19/2023] [Accepted: 12/07/2023] [Indexed: 12/27/2023]
Abstract
This article presents a systematic review aimed at mapping the literature published in the last decade on the use of machine learning (ML) for clinical decision-making through wearable inertial sensors. The review aims to analyze the trends, perspectives, strengths, and limitations of current literature in integrating ML and inertial measurements for clinical applications. The review process involved defining four research questions and applying four relevance assessment indicators to filter the search results, providing insights into the pathologies studied, technologies and setups used, data processing schemes, ML techniques applied, and their clinical impact. When combined with ML techniques, inertial measurement units (IMUs) have primarily been utilized to detect and classify diseases and their associated motor symptoms. They have also been used to monitor changes in movement patterns associated with the presence, severity, and progression of pathology across a diverse range of clinical conditions. ML models trained with IMU data have shown potential in improving patient care by objectively classifying and predicting motor symptoms, often with a minimally encumbering setup. The findings contribute to understanding the current state of ML integration with wearable inertial sensors in clinical practice and identify future research directions. Despite the widespread adoption of these technologies and techniques in clinical applications, there is still a need to translate them into routine clinical practice. This underscores the importance of fostering a closer collaboration between technological experts and professionals in the medical field.
Collapse
Affiliation(s)
- Daniele Bibbo
- Department of Industrial, Electronic and Mechanical Engineering, Roma Tre University, Rome, Italy
| | | | - Maurizio Schmid
- Department of Industrial, Electronic and Mechanical Engineering, Roma Tre University, Rome, Italy
| | - Simone Ranaldi
- Department of Industrial, Electronic and Mechanical Engineering, Roma Tre University, Rome, Italy
| |
Collapse
|
3
|
Xu Z, Shen B, Tang Y, Wu J, Wang J. Deep Clinical Phenotyping of Parkinson's Disease: Towards a New Era of Research and Clinical Care. PHENOMICS (CHAM, SWITZERLAND) 2022; 2:349-361. [PMID: 36939759 PMCID: PMC9590510 DOI: 10.1007/s43657-022-00051-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/24/2021] [Revised: 03/12/2022] [Accepted: 03/28/2022] [Indexed: 11/27/2022]
Abstract
Despite recent advances in technology, clinical phenotyping of Parkinson's disease (PD) has remained relatively limited as current assessments are mainly based on empirical observation and subjective categorical judgment at the clinic. A lack of comprehensive, objective, and quantifiable clinical phenotyping data has hindered our capacity to diagnose, assess patients' conditions, discover pathogenesis, identify preclinical stages and clinical subtypes, and evaluate new therapies. Therefore, deep clinical phenotyping of PD patients is a necessary step towards understanding PD pathology and improving clinical care. In this review, we present a growing community consensus and perspective on how to clinically phenotype this disease, that is, to phenotype the entire course of disease progression by integrating capacity, performance, and perception approaches with state-of-the-art technology. We also explore the most studied aspects of PD deep clinical phenotypes, namely, bradykinesia, tremor, dyskinesia and motor fluctuation, gait impairment, speech impairment, and non-motor phenotypes.
Collapse
Affiliation(s)
- Zhiheng Xu
- Department of Neurology and National Research Center for Aging and Medicine & National Center for Neurological Disorders, State Key Laboratory of Medical Neurobiology, Huashan Hospital, Fudan University, Shanghai, 200040 China
| | - Bo Shen
- Department of Neurology and National Research Center for Aging and Medicine & National Center for Neurological Disorders, State Key Laboratory of Medical Neurobiology, Huashan Hospital, Fudan University, Shanghai, 200040 China
| | - Yilin Tang
- Department of Neurology and National Research Center for Aging and Medicine & National Center for Neurological Disorders, State Key Laboratory of Medical Neurobiology, Huashan Hospital, Fudan University, Shanghai, 200040 China
| | - Jianjun Wu
- Department of Neurology and National Research Center for Aging and Medicine & National Center for Neurological Disorders, State Key Laboratory of Medical Neurobiology, Huashan Hospital, Fudan University, Shanghai, 200040 China
| | - Jian Wang
- Department of Neurology and National Research Center for Aging and Medicine & National Center for Neurological Disorders, State Key Laboratory of Medical Neurobiology, Huashan Hospital, Fudan University, Shanghai, 200040 China
| |
Collapse
|
4
|
Ma C, Zhang P, Wang J, Zhang J, Pan L, Li X, Yin C, Li A, Zong R, Zhang Z. Objective quantification of the severity of postural tremor based on kinematic parameters: A multi-sensory fusion study. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2022; 219:106741. [PMID: 35338882 DOI: 10.1016/j.cmpb.2022.106741] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/27/2021] [Revised: 02/27/2022] [Accepted: 03/08/2022] [Indexed: 06/14/2023]
Abstract
BACKGROUND Current clinical assessments of essential tremor (ET) are primarily based on expert consultation combined with reviewing patient complaints, physician expertise, and diagnostic experience. Thus, traditional evaluation methods often lead to biased diagnostic results. There is a clinical demand for a method that can objectively quantify the severity of the patient's disease. METHODS This study aims to develop an artificial intelligence-aided diagnosis method based on multi-sensory fusion wearables. The experiment relies on a rigorous clinical trial paradigm to collect multi-modal fusion of signals from 98 ET patients. At the same time, three clinicians scored independently, and the consensus score obtained was used as the ground truth for the machine learning models. RESULTS Sixty kinematic parameters were extracted from the signals recorded by the nine-axis inertial measurement unit (IMU). The results showed that most of the features obtained by IMU could effectively characterize the severity of the tremors. The accuracy of the optimal model for three tasks classifying five severity levels was 97.71%, 97.54%, and 97.72%, respectively. CONCLUSIONS This paper reports the first attempt to combine multiple feature selection and machine learning algorithms for fine-grained automatic quantification of postural tremor in ET patients. The promising results showed the potential of the proposed approach to quantify the severity of ET objectively.
Collapse
Affiliation(s)
- Chenbin Ma
- Center for Artificial Intelligence in Medicine, Medical Innovation Research Department, PLA General Hospital, 100853, Beijing, China; Beijing Advanced Innovation Center for Biomedical Engineering, Beihang University, 100191, Beijing, China; School of Biological Science and Medical Engineering, Beihang University, 100191, Beijing, China; Shenyuan Honors College, Beihang University, 100191, Beijing, China
| | - Peng Zhang
- Beijing Advanced Innovation Center for Biomedical Engineering, Beihang University, 100191, Beijing, China; School of Biological Science and Medical Engineering, Beihang University, 100191, Beijing, China
| | - Jiachen Wang
- Medical School of Chinese PLA, 100853, Beijing, China
| | - Jian Zhang
- Medical School of Chinese PLA, 100853, Beijing, China
| | - Longsheng Pan
- Department of Neurosurgery, First Medical Center of Chinese PLA General Hospital, 100853, Beijing, China
| | - Xuemei Li
- Clinics of Cadre, Department of Outpatient, First Medical Center of Chinese PLA General Hospital, 100853, Beijing, China
| | - Chunyu Yin
- Clinics of Cadre, Department of Outpatient, First Medical Center of Chinese PLA General Hospital, 100853, Beijing, China
| | - Ailing Li
- Pusheng Yixin (Beijing) Hospital Management Co., Ltd, 100020, Beijing, China
| | - Rui Zong
- Department of Neurosurgery, First Medical Center of Chinese PLA General Hospital, 100853, Beijing, China.
| | - Zhengbo Zhang
- Center for Artificial Intelligence in Medicine, Medical Innovation Research Department, PLA General Hospital, 100853, Beijing, China.
| |
Collapse
|
5
|
Giannakopoulou KM, Roussaki I, Demestichas K. Internet of Things Technologies and Machine Learning Methods for Parkinson's Disease Diagnosis, Monitoring and Management: A Systematic Review. SENSORS (BASEL, SWITZERLAND) 2022; 22:1799. [PMID: 35270944 PMCID: PMC8915040 DOI: 10.3390/s22051799] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/28/2021] [Revised: 02/17/2022] [Accepted: 02/21/2022] [Indexed: 12/15/2022]
Abstract
Parkinson's disease is a chronic neurodegenerative disease that affects a large portion of the population, especially the elderly. It manifests with motor, cognitive and other types of symptoms, decreasing significantly the patients' quality of life. The recent advances in the Internet of Things and Artificial Intelligence fields, including the subdomains of machine learning and deep learning, can support Parkinson's disease patients, their caregivers and clinicians at every stage of the disease, maximizing the treatment effectiveness and minimizing the respective healthcare costs at the same time. In this review, the considered studies propose machine learning models, trained on data acquired via smart devices, wearable or non-wearable sensors and other Internet of Things technologies, to provide predictions or estimations regarding Parkinson's disease aspects. Seven hundred and seventy studies have been retrieved from three dominant academic literature databases. Finally, one hundred and twelve of them have been selected in a systematic way and have been considered in the state-of-the-art systematic review presented in this paper. These studies propose various methods, applied on various sensory data to address different Parkinson's disease-related problems. The most widely deployed sensors, the most commonly addressed problems and the best performing algorithms are highlighted. Finally, some challenges are summarized along with some future considerations and opportunities that arise.
Collapse
Affiliation(s)
- Konstantina-Maria Giannakopoulou
- School of Electrical and Computer Engineering, National Technical University of Athens, 15773 Athens, Greece; (K.-M.G.); (K.D.)
- Institute of Communication and Computer Systems, 10682 Athens, Greece
| | - Ioanna Roussaki
- School of Electrical and Computer Engineering, National Technical University of Athens, 15773 Athens, Greece; (K.-M.G.); (K.D.)
- Institute of Communication and Computer Systems, 10682 Athens, Greece
| | - Konstantinos Demestichas
- School of Electrical and Computer Engineering, National Technical University of Athens, 15773 Athens, Greece; (K.-M.G.); (K.D.)
- Institute of Communication and Computer Systems, 10682 Athens, Greece
| |
Collapse
|
6
|
Sibley K, Girges C, Candelario J, Milabo C, Salazar M, Esperida JO, Dushin Y, Limousin P, Foltynie T. An Evaluation of KELVIN, an Artificial Intelligence Platform, as an Objective Assessment of the MDS UPDRS Part III. JOURNAL OF PARKINSON'S DISEASE 2022; 12:2223-2233. [PMID: 36155530 DOI: 10.3233/jpd-223493] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/16/2023]
Abstract
BACKGROUND Parkinson's disease severity is typically measured using the Movement Disorder Society Unified Parkinson's disease rating scale (MDS-UPDRS). While training for this scale exists, users may vary in how they score a patient with the consequence of intra-rater and inter-rater variability. OBJECTIVE In this study we explored the consistency of an artificial intelligence platform compared with traditional clinical scoring in the assessment of motor severity in PD. METHODS Twenty-two PD patients underwent simultaneous MDS-UPDRS scoring by two experienced MDS-UPDRS raters and the two sets of accompanying video footage were also scored by an artificial intelligence video analysis platform known as KELVIN. RESULTS KELVIN was able to produce a summary score for 7 MDS-UPDRS part 3 items with good inter-rater reliability (Intraclass Correlation Coefficient (ICC) 0.80 in the OFF-medication state, ICC 0.73 in the ON-medication state). Clinician scores had exceptionally high levels of inter-rater reliability in both the OFF (0.99) and ON (0.94) medication conditions (possibly reflecting the highly experienced team). There was an ICC of 0.84 in the OFF-medication state and 0.31 in the ON-medication state between the mean Clinician and mean Kelvin scores for the equivalent 7 motor items, possibly due to dyskinesia impacting on the KELVIN scores. CONCLUSION We conclude that KELVIN may prove useful in the capture and scoring of multiple items of MDS-UPDRS part 3 with levels of consistency not far short of that achieved by experienced MDS-UPDRS clinical raters, and is worthy of further investigation.
Collapse
Affiliation(s)
- Krista Sibley
- Department of Clinical & Movement Neurosciences, UCL Institute of Neurology, National Hospital for Neurology and Neurosurgery, London, UK
| | - Christine Girges
- Department of Clinical & Movement Neurosciences, UCL Institute of Neurology, National Hospital for Neurology and Neurosurgery, London, UK
| | - Joseph Candelario
- Department of Clinical & Movement Neurosciences, UCL Institute of Neurology, National Hospital for Neurology and Neurosurgery, London, UK
| | - Catherine Milabo
- Department of Clinical & Movement Neurosciences, UCL Institute of Neurology, National Hospital for Neurology and Neurosurgery, London, UK
| | - Maricel Salazar
- Department of Clinical & Movement Neurosciences, UCL Institute of Neurology, National Hospital for Neurology and Neurosurgery, London, UK
| | - John Onil Esperida
- Department of Clinical & Movement Neurosciences, UCL Institute of Neurology, National Hospital for Neurology and Neurosurgery, London, UK
| | | | - Patricia Limousin
- Department of Clinical & Movement Neurosciences, UCL Institute of Neurology, National Hospital for Neurology and Neurosurgery, London, UK
| | - Thomas Foltynie
- Department of Clinical & Movement Neurosciences, UCL Institute of Neurology, National Hospital for Neurology and Neurosurgery, London, UK
| |
Collapse
|
7
|
A dual-branch model for diagnosis of Parkinson’s disease based on the independent and joint features of the left and right gait. APPL INTELL 2021. [DOI: 10.1007/s10489-020-02182-5] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
|
8
|
A patient network-based machine learning model for disease prediction: The case of type 2 diabetes mellitus. APPL INTELL 2021. [DOI: 10.1007/s10489-021-02533-w] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/11/2022]
|
9
|
Hosseini MS, Moradi MH. Adaptive fuzzy-SIFT rule-based registration for 3D cardiac motion estimation. APPL INTELL 2021. [DOI: 10.1007/s10489-021-02430-2] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/01/2022]
|
10
|
Hssayeni MD, Jimenez-Shahed J, Burack MA, Ghoraani B. Ensemble deep model for continuous estimation of Unified Parkinson's Disease Rating Scale III. Biomed Eng Online 2021; 20:32. [PMID: 33789666 PMCID: PMC8010504 DOI: 10.1186/s12938-021-00872-w] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/08/2021] [Accepted: 03/18/2021] [Indexed: 12/27/2022] Open
Abstract
BACKGROUND Unified Parkinson Disease Rating Scale-part III (UPDRS III) is part of the standard clinical examination performed to track the severity of Parkinson's disease (PD) motor complications. Wearable technologies could be used to reduce the need for on-site clinical examinations of people with Parkinson's disease (PwP) and provide a reliable and continuous estimation of the severity of PD at home. The reported estimation can be used to successfully adjust the dose and interval of PD medications. METHODS We developed a novel algorithm for unobtrusive and continuous UPDRS-III estimation at home using two wearable inertial sensors mounted on the wrist and ankle. We used the ensemble of three deep-learning models to detect UPDRS-III-related patterns from a combination of hand-crafted features, raw temporal signals, and their time-frequency representation. Specifically, we used a dual-channel, Long Short-Term Memory (LSTM) for hand-crafted features, 1D Convolutional Neural Network (CNN)-LSTM for raw signals, and 2D CNN-LSTM for time-frequency data. We utilized transfer learning from activity recognition data and proposed a two-stage training for the CNN-LSTM networks to cope with the limited amount of data. RESULTS The algorithm was evaluated on gyroscope data from 24 PwP as they performed different daily living activities. The estimated UPDRS-III scores had a correlation of [Formula: see text] and a mean absolute error of 5.95 with the clinical examination scores without requiring the patients to perform any specific tasks. CONCLUSION Our analysis demonstrates the potential of our algorithm for estimating PD severity scores unobtrusively at home. Such an algorithm could provide the required motor-complication measurements without unnecessary clinical visits and help the treating physician provide effective management of the disease.
Collapse
Affiliation(s)
- Murtadha D Hssayeni
- Department of Computer and Electrical Engineering and Computer Science, Florida Atlantic University, Boca Raton, FL, 33431, USA
| | | | - Michelle A Burack
- Department of Neurology, University of Rochester Medical Center, Rochester, NY, USA
| | - Behnaz Ghoraani
- Department of Computer and Electrical Engineering and Computer Science, Florida Atlantic University, Boca Raton, FL, 33431, USA.
| |
Collapse
|
11
|
|