1
|
Davidashvilly S, Cardei M, Hssayeni M, Chi C, Ghoraani B. Deep neural networks for wearable sensor-based activity recognition in Parkinson's disease: investigating generalizability and model complexity. Biomed Eng Online 2024; 23:17. [PMID: 38336781 PMCID: PMC10858599 DOI: 10.1186/s12938-024-01214-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2023] [Accepted: 01/25/2024] [Indexed: 02/12/2024] Open
Abstract
BACKGROUND The research gap addressed in this study is the applicability of deep neural network (NN) models on wearable sensor data to recognize different activities performed by patients with Parkinson's Disease (PwPD) and the generalizability of these models to PwPD using labeled healthy data. METHODS The experiments were carried out utilizing three datasets containing wearable motion sensor readings on common activities of daily living. The collected readings were from two accelerometer sensors. PAMAP2 and MHEALTH are publicly available datasets collected from 10 and 9 healthy, young subjects, respectively. A private dataset of a similar nature collected from 14 PwPD patients was utilized as well. Deep NN models were implemented with varying levels of complexity to investigate the impact of data augmentation, manual axis reorientation, model complexity, and domain adaptation on activity recognition performance. RESULTS A moderately complex model trained on the augmented PAMAP2 dataset and adapted to the Parkinson domain using domain adaptation achieved the best activity recognition performance with an accuracy of 73.02%, which was significantly higher than the accuracy of 63% reported in previous studies. The model's F1 score of 49.79% significantly improved compared to the best cross-testing of 33.66% F1 score with only data augmentation and 2.88% F1 score without data augmentation or domain adaptation. CONCLUSION These findings suggest that deep NN models originating on healthy data have the potential to recognize activities performed by PwPD accurately and that data augmentation and domain adaptation can improve the generalizability of models in the healthy-to-PwPD transfer scenario. The simple/moderately complex architectures tested in this study could generalize better to the PwPD domain when trained on a healthy dataset compared to the most complex architectures used. The findings of this study could contribute to the development of accurate wearable-based activity monitoring solutions for PwPD, improving clinical decision-making and patient outcomes based on patient activity levels.
Collapse
Affiliation(s)
- Shelly Davidashvilly
- Electrical and Computer Engineering, Florida Atlantic University, Boca Raton, FL, 33431, US
| | - Maria Cardei
- Electrical and Computer Engineering, Florida Atlantic University, Boca Raton, FL, 33431, US
- Biomedical Engineering, University of Florida, Gainesville, FL, US
| | - Murtadha Hssayeni
- Electrical and Computer Engineering, Florida Atlantic University, Boca Raton, FL, 33431, US
- Computer Engineering, University of Technology, Baghdad, Iraq
| | - Christopher Chi
- Electrical and Computer Engineering, Florida Atlantic University, Boca Raton, FL, 33431, US
| | - Behnaz Ghoraani
- Electrical and Computer Engineering, Florida Atlantic University, Boca Raton, FL, 33431, US.
| |
Collapse
|
2
|
Fatima R, Khan MH, Nisar MA, Doniec R, Farid MS, Grzegorzek M. A Systematic Evaluation of Feature Encoding Techniques for Gait Analysis Using Multimodal Sensory Data. SENSORS (BASEL, SWITZERLAND) 2023; 24:75. [PMID: 38202937 PMCID: PMC10780594 DOI: 10.3390/s24010075] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/15/2023] [Revised: 12/09/2023] [Accepted: 12/19/2023] [Indexed: 01/12/2024]
Abstract
This paper addresses the problem of feature encoding for gait analysis using multimodal time series sensory data. In recent years, the dramatic increase in the use of numerous sensors, e.g., inertial measurement unit (IMU), in our daily wearable devices has gained the interest of the research community to collect kinematic and kinetic data to analyze the gait. The most crucial step for gait analysis is to find the set of appropriate features from continuous time series data to accurately represent human locomotion. This paper presents a systematic assessment of numerous feature extraction techniques. In particular, three different feature encoding techniques are presented to encode multimodal time series sensory data. In the first technique, we utilized eighteen different handcrafted features which are extracted directly from the raw sensory data. The second technique follows the Bag-of-Visual-Words model; the raw sensory data are encoded using a pre-computed codebook and a locality-constrained linear encoding (LLC)-based feature encoding technique. We evaluated two different machine learning algorithms to assess the effectiveness of the proposed features in the encoding of raw sensory data. In the third feature encoding technique, we proposed two end-to-end deep learning models to automatically extract the features from raw sensory data. A thorough experimental evaluation is conducted on four large sensory datasets and their outcomes are compared. A comparison of the recognition results with current state-of-the-art methods demonstrates the computational efficiency and high efficacy of the proposed feature encoding method. The robustness of the proposed feature encoding technique is also evaluated to recognize human daily activities. Additionally, this paper also presents a new dataset consisting of the gait patterns of 42 individuals, gathered using IMU sensors.
Collapse
Affiliation(s)
- Rimsha Fatima
- Department of Computer Science, University of the Punjab, Lahore 54590, Pakistan (M.S.F.)
| | - Muhammad Hassan Khan
- Department of Computer Science, University of the Punjab, Lahore 54590, Pakistan (M.S.F.)
| | - Muhammad Adeel Nisar
- Department of Information Technology, University of the Punjab, Lahore 54000, Pakistan;
| | - Rafał Doniec
- Faculty of Biomedical Engineering, The Silesian University of Technology, 44-100 Gliwice, Poland;
| | - Muhammad Shahid Farid
- Department of Computer Science, University of the Punjab, Lahore 54590, Pakistan (M.S.F.)
| | - Marcin Grzegorzek
- Institute of Medical Informatics, University of Lübeck, 23562 Lübeck, Germany
| |
Collapse
|
3
|
Morgan C, Tonkin EL, Masullo A, Jovan F, Sikdar A, Khaire P, Mirmehdi M, McConville R, Tourte GJL, Whone A, Craddock I. A multimodal dataset of real world mobility activities in Parkinson's disease. Sci Data 2023; 10:918. [PMID: 38123584 PMCID: PMC10733419 DOI: 10.1038/s41597-023-02663-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2023] [Accepted: 10/19/2023] [Indexed: 12/23/2023] Open
Abstract
Parkinson's disease (PD) is a neurodegenerative disorder characterised by motor symptoms such as gait dysfunction and postural instability. Technological tools to continuously monitor outcomes could capture the hour-by-hour symptom fluctuations of PD. Development of such tools is hampered by the lack of labelled datasets from home settings. To this end, we propose REMAP (REal-world Mobility Activities in Parkinson's disease), a human rater-labelled dataset collected in a home-like setting. It includes people with and without PD doing sit-to-stand transitions and turns in gait. These discrete activities are captured from periods of free-living (unobserved, unstructured) and during clinical assessments. The PD participants withheld their dopaminergic medications for a time (causing increased symptoms), so their activities are labelled as being "on" or "off" medications. Accelerometry from wrist-worn wearables and skeleton pose video data is included. We present an open dataset, where the data is coarsened to reduce re-identifiability, and a controlled dataset available on application which contains more refined data. A use-case for the data to estimate sit-to-stand speed and duration is illustrated.
Collapse
Affiliation(s)
- Catherine Morgan
- Movement Disorders Group, Bristol Brain Centre, North Bristol NHS Trust, Southmead Hospital, Southmead Road, Bristol, BS10 5NB, UK
- Translational Health Sciences, University of Bristol, 5 Tyndall Ave, Bristol, BS8 1UD, UK
| | - Emma L Tonkin
- Faculty of Engineering, University of Bristol, Digital Health Offices, 1 Cathedral Square, Bristol, BS1 5DD, UK.
| | - Alessandro Masullo
- Faculty of Engineering, University of Bristol, Digital Health Offices, 1 Cathedral Square, Bristol, BS1 5DD, UK
| | - Ferdian Jovan
- Faculty of Engineering, University of Bristol, Digital Health Offices, 1 Cathedral Square, Bristol, BS1 5DD, UK
- School of Natural and Computing Sciences, University of Aberdeen, Aberdeen, UK
| | - Arindam Sikdar
- Faculty of Engineering, University of Bristol, Digital Health Offices, 1 Cathedral Square, Bristol, BS1 5DD, UK
- Edge Hill University, Ormskirk, UK
| | - Pushpajit Khaire
- Faculty of Engineering, University of Bristol, Digital Health Offices, 1 Cathedral Square, Bristol, BS1 5DD, UK
- Datta Meghe Institute of Higher Education and Research, Wardha, India
| | - Majid Mirmehdi
- Faculty of Engineering, University of Bristol, Digital Health Offices, 1 Cathedral Square, Bristol, BS1 5DD, UK
| | - Ryan McConville
- Faculty of Engineering, University of Bristol, Digital Health Offices, 1 Cathedral Square, Bristol, BS1 5DD, UK
| | - Gregory J L Tourte
- Faculty of Engineering, University of Bristol, Digital Health Offices, 1 Cathedral Square, Bristol, BS1 5DD, UK
- Advanced Research Computing, University of Oxford, Oxford, UK
| | - Alan Whone
- Movement Disorders Group, Bristol Brain Centre, North Bristol NHS Trust, Southmead Hospital, Southmead Road, Bristol, BS10 5NB, UK
- Translational Health Sciences, University of Bristol, 5 Tyndall Ave, Bristol, BS8 1UD, UK
| | - Ian Craddock
- Faculty of Engineering, University of Bristol, Digital Health Offices, 1 Cathedral Square, Bristol, BS1 5DD, UK
| |
Collapse
|