Deep Learning Approaches for Detecting Freezing of Gait in Parkinson's Disease Patients through On-Body Acceleration Sensors

Insights into Parkinson's Disease-Related Freezing of Gait Detection and Prediction Approaches: A Meta Analysis

Parkinson's Disease (PD) is a complex neurodegenerative disorder characterized by a spectrum of motor and non-motor symptoms, prominently featuring the freezing of gait (FOG), which significantly impairs patients' quality of life. Despite extensive research, the precise mechanisms underlying FOG remain elusive, posing challenges for effective management and treatment. This paper presents a comprehensive meta-analysis of FOG prediction and detection methodologies, with a focus on the integration of wearable sensor technology and machine learning (ML) approaches. Through an exhaustive review of the literature, this study identifies key trends, datasets, preprocessing techniques, feature extraction methods, evaluation metrics, and comparative analyses between ML and non-ML approaches. The analysis also explores the utilization of cueing devices. The limited adoption of explainable AI (XAI) approaches in FOG prediction research represents a significant gap. Improving user acceptance and comprehension requires an understanding of the logic underlying algorithm predictions. Current FOG detection and prediction research has a number of limitations, which are identified in the discussion. These include issues with cueing devices, dataset constraints, ethical and privacy concerns, financial and accessibility restrictions, and the requirement for multidisciplinary collaboration. Future research avenues center on refining explainability, expanding and diversifying datasets, adhering to user requirements, and increasing detection and prediction accuracy. The findings contribute to advancing the understanding of FOG and offer valuable guidance for the development of more effective detection and prediction methodologies, ultimately benefiting individuals affected by PD.

A machine learning contest enhances automated freezing of gait detection and reveals time-of-day effects

Freezing of gait (FOG) is a debilitating problem that markedly impairs the mobility and independence of 38-65% of people with Parkinson's disease. During a FOG episode, patients report that their feet are suddenly and inexplicably "glued" to the floor. The lack of a widely applicable, objective FOG detection method obstructs research and treatment. To address this problem, we organized a 3-month machine-learning contest, inviting experts from around the world to develop wearable sensor-based FOG detection algorithms. 1,379 teams from 83 countries submitted 24,862 solutions. The winning solutions demonstrated high accuracy, high specificity, and good precision in FOG detection, with strong correlations to gold-standard references. When applied to continuous 24/7 data, the solutions revealed previously unobserved patterns in daily living FOG occurrences. This successful endeavor underscores the potential of machine learning contests to rapidly engage AI experts in addressing critical medical challenges and provides a promising means for objective FOG quantification.

Parkinson's disease diagnosis using deep learning: A bibliometric analysis and literature review

Parkinson's Disease (PD) is a progressive neurodegenerative illness triggered by decreased dopamine secretion. Deep Learning (DL) has gained substantial attention in PD diagnosis research, with an increase in the number of published papers in this discipline. PD detection using DL has presented more promising outcomes as compared with common machine learning approaches. This article aims to conduct a bibliometric analysis and a literature review focusing on the prominent developments taking place in this area. To achieve the target of the study, we retrieved and analyzed the available research papers in the Scopus database. Following that, we conducted a bibliometric analysis to inspect the structure of keywords, authors, and countries in the surveyed studies by providing visual representations of the bibliometric data using VOSviewer software. The study also provides an in-depth review of the literature focusing on different indicators of PD, deployed approaches, and performance metrics. The outcomes indicate the firm development of PD diagnosis using DL approaches over time and a large diversity of studies worldwide. Additionally, the literature review presented a research gap in DL approaches related to incremental learning, particularly in relation to big data analysis.

Detection and prediction of freezing of gait with wearable sensors in Parkinson's disease

Freezing of gait (FoG) is one of the most distressing symptoms of Parkinson's Disease (PD), commonly occurring in patients at middle and late stages of the disease. Automatic and accurate FoG detection and prediction have emerged as a promising tool for long-term monitoring of PD and implementation of gait assistance systems. This paper reviews the recent development of FoG detection and prediction using wearable sensors, with attention on identifying knowledge gaps that need to be filled in future research. This review searched the PubMed and Web of Science databases to collect studies that detect or predict FoG with wearable sensors. After screening, 89 of 270 articles were included. The data description, extracted features, detection/prediction methods, and classification performance were extracted from the articles. As the number of papers of this area is increasing, the performance has been steadily improved. However, small datasets and inconsistent evaluation processes still hinder the application of FoG detection and prediction with wearable sensors in clinical practice.

Machine learning to detect, stage and classify diseases and their symptoms based on inertial sensor data: a mapping review

This article presents a systematic review aimed at mapping the literature published in the last decade on the use of machine learning (ML) for clinical decision-making through wearable inertial sensors. The review aims to analyze the trends, perspectives, strengths, and limitations of current literature in integrating ML and inertial measurements for clinical applications. The review process involved defining four research questions and applying four relevance assessment indicators to filter the search results, providing insights into the pathologies studied, technologies and setups used, data processing schemes, ML techniques applied, and their clinical impact. When combined with ML techniques, inertial measurement units (IMUs) have primarily been utilized to detect and classify diseases and their associated motor symptoms. They have also been used to monitor changes in movement patterns associated with the presence, severity, and progression of pathology across a diverse range of clinical conditions. ML models trained with IMU data have shown potential in improving patient care by objectively classifying and predicting motor symptoms, often with a minimally encumbering setup. The findings contribute to understanding the current state of ML integration with wearable inertial sensors in clinical practice and identify future research directions. Despite the widespread adoption of these technologies and techniques in clinical applications, there is still a need to translate them into routine clinical practice. This underscores the importance of fostering a closer collaboration between technological experts and professionals in the medical field.

Comparison of state-of-the-art deep learning architectures for detection of freezing of gait in Parkinson's disease

Introduction

Freezing of gait (FOG) is one of the most debilitating motor symptoms experienced by patients with Parkinson's disease (PD). FOG detection is possible using acceleration data from wearable sensors, and a convolutional neural network (CNN) is often used to determine the presence of FOG epochs. We compared the performance of a standard CNN for the detection of FOG with two more complex networks, which are well suited for time series data, the MiniRocket and the InceptionTime.

Methods

We combined acceleration data of people with PD across four studies. The final data set was split into a training (80%) and hold-out test (20%) set. A fifth study was included as an unseen test set. The data were windowed (2 s) and five-fold cross-validation was applied. The CNN, MiniRocket, and InceptionTime models were evaluated using a receiver operating characteristic (ROC) curve and its area under the curve (AUC). Multiple sensor configurations were evaluated for the best model. The geometric mean was subsequently calculated to select the optimal threshold. The selected model and threshold were evaluated on the hold-out and unseen test set.

Results

A total of 70 participants (23.7 h, 9% FOG) were included in this study for training and testing, and in addition, 10 participants provided an unseen test set (2.4 h, 11% FOG). The CNN performed best (AUC = 0.86) in comparison to the InceptionTime (AUC = 0.82) and MiniRocket (AUC = 0.76) models. For the CNN, we found a similar performance for a seven-sensor configuration (lumbar, upper and lower legs and feet; AUC = 0.86), six-sensor configuration (upper and lower legs and feet; AUC = 0.87), and two-sensor configuration (lower legs; AUC = 0.86). The optimal threshold of 0.45 resulted in a sensitivity of 77% and a specificity of 58% for the hold-out set (AUC = 0.72), and a sensitivity of 85% and a specificity of 68% for the unseen test set (AUC = 0.90).

Conclusion

We confirmed that deep learning can be used to detect FOG in a large, heterogeneous dataset. The CNN model outperformed more complex networks. This model could be employed in future personalized interventions, with the ultimate goal of using automated FOG detection to trigger real-time cues to alleviate FOG in daily life.

Context Recognition Algorithms for Energy-Efficient Freezing-of-Gait Detection in Parkinson's Disease

Freezing of gait (FoG) is a disabling clinical phenomenon of Parkinson's disease (PD) characterized by the inability to move the feet forward despite the intention to walk. It is one of the most troublesome symptoms of PD, leading to an increased risk of falls and reduced quality of life. The combination of wearable inertial sensors and machine learning (ML) algorithms represents a feasible solution to monitor FoG in real-world scenarios. However, traditional FoG detection algorithms process all data indiscriminately without considering the context of the activity during which FoG occurs. This study aimed to develop a lightweight, context-aware algorithm that can activate FoG detection systems only under certain circumstances, thus reducing the computational burden. Several approaches were implemented, including ML and deep learning (DL) gait recognition methods, as well as a single-threshold method based on acceleration magnitude. To train and evaluate the context algorithms, data from a single inertial sensor were extracted using three different datasets encompassing a total of eighty-one PD patients. Sensitivity and specificity for gait recognition ranged from 0.95 to 0.96 and 0.80 to 0.93, respectively, with the one-dimensional convolutional neural network providing the best results. The threshold approach performed better than ML- and DL-based methods when evaluating the effect of context awareness on FoG detection performance. Overall, context algorithms allow for discarding more than 55% of non-FoG data and less than 4% of FoG episodes. The results indicate that a context classifier can reduce the computational burden of FoG detection algorithms without significantly affecting the FoG detection rate. Thus, implementation of context awareness can present an energy-efficient solution for long-term FoG monitoring in ambulatory and free-living settings.

Real-time detection of freezing of gait in Parkinson's disease using multi-head convolutional neural networks and a single inertial sensor

BACKGROUND

Freezing of gait (FOG) is one of the most disabling symptoms of Parkinson's disease (PD), contributing to poor quality of life and increased risk of falls. Wearable sensors represent a valuable means for detecting FOG in the home environment. Moreover, real-time feedback has proven to help reduce the duration of FOG episodes. This work proposes a robust real-time FOG detection algorithm, which is easy to implement in stand-alone devices working in non-supervised conditions.

METHOD

Data from three different data sets were used in this study, with two employed as independent test sets. Acceleration recordings from 118 PD patients and 21 healthy elderly subjects were collected while they performed simulated daily living activities. A single inertial sensor was attached to the waist of each subject. More than 17 h of valid data and a total number of 1110 FOG episodes were analyzed in this study. The implemented algorithm consisted of a multi-head convolutional neural network, which exploited different spatial resolutions in the analysis of inertial data. The architecture and the model parameters were designed to provide optimal performance while reducing computational complexity and testing time.

RESULTS

The developed algorithm demonstrated good to excellent classification performance, with more than 50% (30%) of FOG episodes predicted on average 3.1 s (1.3 s) before the actual onset in the main (independent) data set. Around 50% of FOG was detected with an average delay of 0.8 s (1.1 s) in the main (independent) data set. Moreover, a specificity above 88% (93%) was obtained when testing the algorithm on the main (independent) test set, while 100% specificity was obtained on healthy elderly subjects.

CONCLUSION

The algorithm proved robust, with low computational complexity and processing time, thus paving the way to a real-time implementation in a stand-alone device that can be used in non-supervised environments.

Recent trends in wearable device used to detect freezing of gait and falls in people with Parkinson's disease: A systematic review

Background

The occurrence of freezing of gait (FOG) is often observed in moderate to last-stage Parkinson's disease (PD), leading to a high risk of falls. The emergence of the wearable device has offered the possibility of FOG detection and falls of patients with PD allowing high validation in a low-cost way.

Objective

This systematic review seeks to provide a comprehensive overview of existing literature to establish the forefront of sensors type, placement and algorithm to detect FOG and falls among patients with PD.

Methods

Two electronic databases were screened by title and abstract to summarize the state of art on FOG and fall detection with any wearable technology among patients with PD. To be eligible for inclusion, papers were required to be full-text articles published in English, and the last search was completed on September 26, 2022. Studies were excluded if they; (i) only examined cueing function for FOG, (ii) only used non-wearable devices to detect or predict FOG or falls, and (iii) did not provide sufficient details about the study design and results. A total of 1,748 articles were retrieved from two databases. However, only 75 articles were deemed to meet the inclusion criteria according to the title, abstract and full-text reviewed. Variable was extracted from chosen research, including authorship, details of the experimental object, type of sensor, device location, activities, year of publication, evaluation in real-time, the algorithm and detection performance.

Results

A total of 72 on FOG detection and 3 on fall detection were selected for data extraction. There were wide varieties of the studied population (from 1 to 131), type of sensor, placement and algorithm. The thigh and ankle were the most popular device location, and the combination of accelerometer and gyroscope was the most frequently used inertial measurement unit (IMU). Furthermore, 41.3% of the studies used the dataset as a resource to examine the validity of their algorithm. The results also showed that increasingly complex machine-learning algorithms had become the trend in FOG and fall detection.

Conclusion

These data support the application of the wearable device to access FOG and falls among patients with PD and controls. Machine learning algorithms and multiple types of sensors have become the recent trend in this field. Future work should consider an adequate sample size, and the experiment should be performed in a free-living environment. Moreover, a consensus on provoking FOG/fall, methods of assessing validity and algorithm are necessary.Systematic Review Registration: PROSPERO, identifier CRD42022370911.

Digital Devices for Assessing Motor Functions in Mobility-Impaired and Healthy Populations: Systematic Literature Review

Background

With the advent of smart sensing technology, mobile and wearable devices can provide continuous and objective monitoring and assessment of motor function outcomes.

Objective

We aimed to describe the existing scientific literature on wearable and mobile technologies that are being used or tested for assessing motor functions in mobility-impaired and healthy adults and to evaluate the degree to which these devices provide clinically valid measures of motor function in these populations.

Methods

A systematic literature review was conducted by searching Embase, MEDLINE, CENTRAL (January 1, 2015, to June 24, 2020), the United States and European Union clinical trial registries, and the United States Food and Drug Administration website using predefined study selection criteria. Study selection, data extraction, and quality assessment were performed by 2 independent reviewers.

Results

A total of 91 publications representing 87 unique studies were included. The most represented clinical conditions were Parkinson disease (n=51 studies), followed by stroke (n=5), Huntington disease (n=5), and multiple sclerosis (n=2). A total of 42 motion-detecting devices were identified, and the majority (n=27, 64%) were created for the purpose of health care–related data collection, although approximately 25% were personal electronic devices (eg, smartphones and watches) and 11% were entertainment consoles (eg, Microsoft Kinect or Xbox and Nintendo Wii). The primary motion outcomes were related to gait (n=30), gross motor movements (n=25), and fine motor movements (n=23). As a group, sensor-derived motion data showed a mean sensitivity of 0.83 (SD 7.27), a mean specificity of 0.84 (SD 15.40), a mean accuracy of 0.90 (SD 5.87) in discriminating between diseased individuals and healthy controls, and a mean Pearson r validity coefficient of 0.52 (SD 0.22) relative to clinical measures. We did not find significant differences in the degree of validity between in-laboratory and at-home sensor-based assessments nor between device class (ie, health care–related device, personal electronic devices, and entertainment consoles).

Conclusions

Sensor-derived motion data can be leveraged to classify and quantify disease status for a variety of neurological conditions. However, most of the recent research on digital clinical measures is derived from proof-of-concept studies with considerable variation in methodological approaches, and much of the reviewed literature has focused on clinical validation, with less than one-quarter of the studies performing analytical validation. Overall, future research is crucially needed to further consolidate that sensor-derived motion data may lead to the development of robust and transformative digital measurements intended to predict, diagnose, and quantify neurological disease state and its longitudinal change.

Deep Learning for Daily Monitoring of Parkinson's Disease Outside the Clinic Using Wearable Sensors

Now that wearable sensors have become more commonplace, it is possible to monitor individual healthcare-related activity outside the clinic, unleashing potential for early detection of events in diseases such as Parkinson's disease (PD). However, the unsupervised and "open world" nature of this type of data collection make such applications difficult to develop. In this proof-of-concept study, we used inertial sensor data from Verily Study Watches worn by individuals for up to 23 h per day over several months to distinguish between seven subjects with PD and four without. Since motor-related PD symptoms such as bradykinesia and gait abnormalities typically present when a PD subject is walking, we initially used human activity recognition (HAR) techniques to identify walk-like activity in the unconstrained, unlabeled data. We then used these "walk-like" events to train one-dimensional convolutional neural networks (1D-CNNs) to determine the presence of PD. We report classification accuracies near 90% on single 5-s walk-like events and 100% accuracy when taking the majority vote over single-event classifications that span a duration of one day. Though based on a small cohort, this study shows the feasibility of leveraging unconstrained wearable sensor data to accurately detect the presence or absence of PD.

Detection and assessment of Parkinson's disease based on gait analysis: A survey

Neurological disorders represent one of the leading causes of disability and mortality in the world. Parkinson's Disease (PD), for example, affecting millions of people worldwide is often manifested as impaired posture and gait. These impairments have been used as a clinical sign for the early detection of PD, as well as an objective index for pervasive monitoring of the PD patients in daily life. This review presents the evidence that demonstrates the relationship between human gait and PD, and illustrates the role of different gait analysis systems based on vision or wearable sensors. It also provides a comprehensive overview of the available automatic recognition systems for the detection and management of PD. The intervening measures for improving gait performance are summarized, in which the smart devices for gait intervention are emphasized. Finally, this review highlights some of the new opportunities in detecting, monitoring, and treating of PD based on gait, which could facilitate the development of objective gait-based biomarkers for personalized support and treatment of PD.

Recognition of Freezing of Gait in Parkinson’s Disease Based on Machine Vision

Background

Freezing of gait (FOG) is a common clinical manifestation of Parkinson’s disease (PD), mostly occurring in the intermediate and advanced stages. FOG is likely to cause patients to fall, resulting in fractures, disabilities and even death. Currently, the pathogenesis of FOG is unclear, and FOG detection and screening methods have various defects, including subjectivity, inconvenience, and high cost. Due to limited public healthcare and transportation resources during the COVID-19 pandemic, there are greater inconveniences for PD patients who need diagnosis and treatment.

Objective

A method was established to automatically recognize FOG in PD patients through videos taken by mobile phone, which is time-saving, labor-saving, and low-cost for daily use, which may overcome the above defects. In the future, PD patients can undergo FOG assessment at any time in the home rather than in the hospital.

Methods

In this study, motion features were extracted from timed up and go (TUG) test and the narrow TUG (Narrow) test videos of 50 FOG-PD subjects through a machine learning method; then a motion recognition model to distinguish between walking and turning stages and a model to recognize FOG in these stages were constructed using the XGBoost algorithm. Finally, we combined these three models to form a multi-stage FOG recognition model.

Results

We adopted the leave-one-subject-out (LOSO) method to evaluate model performance, and the multi-stage FOG recognition model achieved a sensitivity of 87.5% sensitivity and a specificity of 79.82%.

Conclusion

A method to realize remote PD patient FOG recognition based on mobile phone video is presented in this paper. This method is convenient with high recognition accuracy and can be used to rapidly evaluate FOG in the home environment and remotely manage FOG-PD, or screen patients in large-scale communities.

Detection of Freezing of Gait Using Convolutional Neural Networks and Data From Lower Limb Motion Sensors

Parkinson's disease (PD) is a chronic, non-reversible neurodegenerative disorder, and freezing of gait (FOG) is one of the most disabling symptoms in PD as it is often the leading cause of falls and injuries that drastically reduces patients' quality of life. In order to monitor continuously and objectively PD patients who suffer from FOG and enable the possibility of on-demand cueing assistance, a sensor-based FOG detection solution can help clinicians manage the disease and help patients overcome freezing episodes. Many recent studies have leveraged deep learning models to detect FOG using signals extracted from inertial measurement unit (IMU) devices. Usually, the latent features and patterns of FOG are discovered from either the time or frequency domain. In this study, we investigated the use of the time-frequency domain by applying the Continuous Wavelet Transform to signals from IMUs placed on the lower limbs of 63 PD patients who suffered from FOG. We built convolutional neural networks to detect the FOG occurrences, and employed the Bayesian Optimisation approach to obtain the hyper-parameters. The results showed that the proposed subject-independent model was able to achieve a geometric mean of 90.7% and a F1 score of 91.5%.

IMU-Based Monitoring for Assistive Diagnosis and Management of IoHT: A Review

With the rapid development of Internet of Things (IoT) technologies, traditional disease diagnoses carried out in medical institutions can now be performed remotely at home or even ambient environments, yielding the concept of the Internet of Health Things (IoHT). Among the diverse IoHT applications, inertial measurement unit (IMU)-based systems play a significant role in the detection of diseases in many fields, such as neurological, musculoskeletal, and mental. However, traditional numerical interpretation methods have proven to be challenging to provide satisfying detection accuracies owing to the low quality of raw data, especially under strong electromagnetic interference (EMI). To address this issue, in recent years, machine learning (ML)-based techniques have been proposed to smartly map IMU-captured data on disease detection and progress. After a decade of development, the combination of IMUs and ML algorithms for assistive disease diagnosis has become a hot topic, with an increasing number of studies reported yearly. A systematic search was conducted in four databases covering the aforementioned topic for articles published in the past six years. Eighty-one articles were included and discussed concerning two aspects: different ML techniques and application scenarios. This review yielded the conclusion that, with the help of ML technology, IMUs can serve as a crucial element in disease diagnosis, severity assessment, characteristic estimation, and monitoring during the rehabilitation process. Furthermore, it summarizes the state-of-the-art, analyzes challenges, and provides foreseeable future trends for developing IMU-ML systems for IoHT.

Recognition of freezing of gait in Parkinson's disease based on combined wearable sensors

Freezing of gait is a common gait disorder among patients with advanced Parkinson’s disease and is associated with falls. This paper designed the relevant experimental procedures to obtain FoG signals from PD patients. Accelerometers, gyroscopes, and force sensing resistor sensors were placed on the lower body of patients. On this basis, the research on the optimal feature extraction method, sensor configuration, and feature quantity selection in the FoG detection process is carried out. Thirteen typical features consisting of time domain, frequency domain and statistical features were extracted from the sensor signals. Firstly, we used the analysis of variance (ANOVA) to select features through comparing the effectiveness of two feature selection methods. Secondly, we evaluated the detection effects with different combinations of sensors to get the best sensors configuration. Finally, we selected the optimal features to construct FoG recognition model based on random forest. After comprehensive consideration of factors such as detection performance, cost, and actual deployment requirements, the 35 features obtained from the left shank gyro and accelerometer, and 78.39% sensitivity, 91.66% specificity, 88.09% accuracy, 77.58% precision and 77.98% f-score were achieved. This objective FoG recognition method has high recognition accuracy, which will be helpful for early FoG symptoms screening and treatment.

Automated freezing of gait assessment with marker-based motion capture and multi-stage spatial-temporal graph convolutional neural networks

BACKGROUND

Freezing of gait (FOG) is a common and debilitating gait impairment in Parkinson's disease. Further insight into this phenomenon is hampered by the difficulty to objectively assess FOG. To meet this clinical need, this paper proposes an automated motion-capture-based FOG assessment method driven by a novel deep neural network.

METHODS

Automated FOG assessment can be formulated as an action segmentation problem, where temporal models are tasked to recognize and temporally localize the FOG segments in untrimmed motion capture trials. This paper takes a closer look at the performance of state-of-the-art action segmentation models when tasked to automatically assess FOG. Furthermore, a novel deep neural network architecture is proposed that aims to better capture the spatial and temporal dependencies than the state-of-the-art baselines. The proposed network, termed multi-stage spatial-temporal graph convolutional network (MS-GCN), combines the spatial-temporal graph convolutional network (ST-GCN) and the multi-stage temporal convolutional network (MS-TCN). The ST-GCN captures the hierarchical spatial-temporal motion among the joints inherent to motion capture, while the multi-stage component reduces over-segmentation errors by refining the predictions over multiple stages. The proposed model was validated on a dataset of fourteen freezers, fourteen non-freezers, and fourteen healthy control subjects.

RESULTS

The experiments indicate that the proposed model outperforms four state-of-the-art baselines. Moreover, FOG outcomes derived from MS-GCN predictions had an excellent (r = 0.93 [0.87, 0.97]) and moderately strong (r = 0.75 [0.55, 0.87]) linear relationship with FOG outcomes derived from manual annotations.

CONCLUSIONS

The proposed MS-GCN may provide an automated and objective alternative to labor-intensive clinician-based FOG assessment. Future work is now possible that aims to assess the generalization of MS-GCN to a larger and more varied verification cohort.

Co-evolution of machine learning and digital technologies to improve monitoring of Parkinson's disease motor symptoms

Parkinson’s disease (PD) is a neurodegenerative disorder characterized by motor impairments such as tremor, bradykinesia, dyskinesia, and gait abnormalities. Current protocols assess PD symptoms during clinic visits and can be subjective. Patient diaries can help clinicians evaluate at-home symptoms, but can be incomplete or inaccurate. Therefore, researchers have developed in-home automated methods to monitor PD symptoms to enable data-driven PD diagnosis and management. We queried the US National Library of Medicine PubMed database to analyze the progression of the technologies and computational/machine learning methods used to monitor common motor PD symptoms. A sub-set of roughly 12,000 papers was reviewed that best characterized the machine learning and technology timelines that manifested from reviewing the literature. The technology used to monitor PD motor symptoms has advanced significantly in the past five decades. Early monitoring began with in-lab devices such as needle-based EMG, transitioned to in-lab accelerometers/gyroscopes, then to wearable accelerometers/gyroscopes, and finally to phone and mobile & web application-based in-home monitoring. Significant progress has also been made with respect to the use of machine learning algorithms to classify PD patients. Using data from different devices (e.g., video cameras, phone-based accelerometers), researchers have designed neural network and non-neural network-based machine learning algorithms to categorize PD patients across tremor, gait, bradykinesia, and dyskinesia. The five-decade co-evolution of technology and computational techniques used to monitor PD motor symptoms has driven significant progress that is enabling the shift from in-lab/clinic to in-home monitoring of PD symptoms.

Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advances

Mobile and wearable devices have enabled numerous applications, including activity tracking, wellness monitoring, and human-computer interaction, that measure and improve our daily lives. Many of these applications are made possible by leveraging the rich collection of low-power sensors found in many mobile and wearable devices to perform human activity recognition (HAR). Recently, deep learning has greatly pushed the boundaries of HAR on mobile and wearable devices. This paper systematically categorizes and summarizes existing work that introduces deep learning methods for wearables-based HAR and provides a comprehensive analysis of the current advancements, developing trends, and major challenges. We also present cutting-edge frontiers and future directions for deep learning-based HAR.

Assessing inertial measurement unit locations for freezing of gait detection and patient preference

Background

Freezing of gait, a common symptom of Parkinson’s disease, presents as sporadic episodes in which an individual’s feet suddenly feel stuck to the ground. Inertial measurement units (IMUs) promise to enable at-home monitoring and personalization of therapy, but there is a lack of consensus on the number and location of IMUs for detecting freezing of gait. The purpose of this study was to assess IMU sets in the context of both freezing of gait detection performance and patient preference.

Methods

Sixteen people with Parkinson’s disease were surveyed about sensor preferences. Raw IMU data from seven people with Parkinson’s disease, wearing up to eleven sensors, were used to train convolutional neural networks to detect freezing of gait. Models trained with data from different sensor sets were assessed for technical performance; a best technical set and minimal IMU set were identified. Clinical utility was assessed by comparing model- and human-rater-determined percent time freezing and number of freezing events.

Results

The best technical set consisted of three IMUs (lumbar and both ankles, AUROC = 0.83), all of which were rated highly wearable. The minimal IMU set consisted of a single ankle IMU (AUROC = 0.80). Correlations between these models and human raters were good to excellent for percent time freezing (ICC = 0.93, 0.89) and number of freezing events (ICC = 0.95, 0.86) for the best technical set and minimal IMU set, respectively.

Conclusions

Several IMU sets consisting of three IMUs or fewer were highly rated for both technical performance and wearability, and more IMUs did not necessarily perform better in FOG detection. We openly share our data and software to further the development and adoption of a general, open-source model that uses raw signals and a standard sensor set for at-home monitoring of freezing of gait.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12984-022-00992-x.

Metaheuristics with Deep Learning-Enabled Parkinson's Disease Diagnosis and Classification Model

Parkinson's disease (PD) affects the movement of people, including the differences in writing skill, speech, tremor, and stiffness in muscles. It is significant to detect the PD at the initial stages so that the person can live a peaceful life for a longer time period. The serious levels of PD are highly risky as the patients get progressive stiffness, which results in the inability of standing or walking. Earlier studies have focused on the detection of PD effectively using voice and speech exams and writing exams. In this aspect, this study presents an improved sailfish optimization algorithm with deep learning (ISFO-DL) model for PD diagnosis and classification. The presented ISFO-DL technique uses the ISFO algorithm and DL model to determine PD and thereby enhances the survival rate of the person. The presented ISFO is a metaheuristic algorithm, which is inspired by a group of hunting sailfish to determine the optimum solution to the problem. Primarily, the ISFO algorithm is applied to derive an optimal subset of features with a fitness function of maximum classification accuracy. At the same time, the rat swarm optimizer (RSO) with the bidirectional gated recurrent unit (BiGRU) is employed as a classifier to determine the existence of PD. The performance validation of the IFSO-DL model takes place using a benchmark Parkinson's dataset, and the results are inspected under several dimensions. The experimental results highlighted the enhanced classification performance of the ISFO-DL technique, and therefore, the proposed model can be employed for the earlier identification of PD.

Modelling and identification of characteristic kinematic features preceding freezing of gait with convolutional neural networks and layer-wise relevance propagation

BACKGROUND

Although deep neural networks (DNNs) are showing state of the art performance in clinical gait analysis, they are considered to be black-box algorithms. In other words, there is a lack of direct understanding of a DNN's ability to identify relevant features, hindering clinical acceptance. Interpretability methods have been developed to ameliorate this concern by providing a way to explain DNN predictions.

METHODS

This paper proposes the use of an interpretability method to explain DNN decisions for classifying the movement that precedes freezing of gait (FOG), one of the most debilitating symptoms of Parkinson's disease (PD). The proposed two-stage pipeline consists of (1) a convolutional neural network (CNN) to model the reduction of movement present before a FOG episode, and (2) layer-wise relevance propagation (LRP) to visualize the underlying features that the CNN perceives as important to model the pathology. The CNN was trained with the sagittal plane kinematics from a motion capture dataset of fourteen PD patients with FOG. The robustness of the model predictions and learned features was further assessed on fourteen PD patients without FOG and fourteen age-matched healthy controls.

RESULTS

The CNN proved highly accurate in modelling the movement that precedes FOG, with 86.8% of the strides being correctly identified. However, the CNN model was unable to model the movement for one of the seven patients that froze during the protocol. The LRP interpretability case study shows that (1) the kinematic features perceived as most relevant by the CNN are the reduced peak knee flexion and the fixed ankle dorsiflexion during the swing phase, (2) very little relevance for FOG is observed in the PD patients without FOG and the healthy control subjects, and (3) the poor predictive performance of one subject is attributed to the patient's unique and severely flexed gait signature.

CONCLUSIONS

The proposed pipeline can aid clinicians in explaining DNN decisions in clinical gait analysis and aid machine learning practitioners in assessing the generalization of their models by ensuring that the predictions are based on meaningful kinematic features.

Real-Time Detection of Freezing Motions in Parkinson's Patients for Adaptive Gait Phase Synchronous Cueing

Parkinson's disease is the second most common neurodegenerative disease worldwide reducing cognitive and motoric abilities of affected persons. Freezing of Gait (FoG) is one of the severe symptoms that is observed in the late stages of the disease and considerably impairs the mobility of the person and raises the risk of falls. Due to the pathology and heterogeneity of the Parkinsonian gait cycle, especially in the case of freezing episodes, the detection of the gait phases with wearables is challenging in Parkinson's disease. This is addressed by introducing a state-automaton-based algorithm for the detection of the foot's motion phases using a shoe-placed inertial sensor. Machine-learning-based methods are investigated to classify the actual motion phase as normal or FoG-affected and to predict the outcome for the next motion phase. For this purpose, spatio-temporal gait and signal parameters are determined from the segmented movement phases. In this context, inertial sensor fusion is applied to the foot's 3D acceleration and rate of turn. Support Vector Machine (SVM) and AdaBoost classifiers have been trained on the data of 16 Parkinson's patients who had shown FoG episodes during a clinical freezing-provoking assessment course. Two clinical experts rated the video-recorded trials and marked episodes with festination, shank trembling, shuffling, or akinesia. Motion phases inside such episodes were labeled as FoG-affected. The classifiers were evaluated using leave-one-patient-out cross-validation. No statistically significant differences could be observed between the different classifiers for FoG detection (p>0.05). An SVM model with 10 features of the actual and two preceding motion phases achieved the highest average performance with 88.5 ± 5.8% sensitivity, 83.3 ± 17.1% specificity, and 92.8 ± 5.9% Area Under the Curve (AUC). The performance of predicting the behavior of the next motion phase was significantly lower compared to the detection classifiers. No statistically significant differences were found between all prediction models. An SVM-predictor with features from the two preceding motion phases had with 81.6 ± 7.7% sensitivity, 70.3 ± 18.4% specificity, and 82.8 ± 7.1% AUC the best average performance. The developed methods enable motion-phase-based FoG detection and prediction and can be utilized for closed-loop systems that provide on-demand gait-phase-synchronous cueing to mitigate FoG symptoms and to prevent complete motoric blockades.

A Study on the Application of LSTM to Judge Bike Accidents for Inflating Wearable Airbags

The traffic accident occurrence rate is increasing relative to the increase in the number of people using personal mobility device (PM). This paper proposes an airbag system with a more efficient algorithm to decide the deployment of a wearable bike airbag in case of an accident. The existing wearable airbags are operated by judging the accident situations using the thresholds of sensors. However, in this case, the judgment accuracy can drop against various motions. This study used the long short-term memory (LSTM) model using the sensor values of the inertial measurement unit (IMU) as input values to judge accident occurrences, which obtains data in real time from the three acceleration-axis and three angular velocity-axis sensors on the driver motion states and judges whether or not an accident has occurred using the obtained data. The existing neural network (NN) or convolutional neural network (CNN) model judges only the input data. This study confirmed that this model has a higher judgment accuracy than the existing NN or CNN by giving strong points even in "past information" through LSTM by regarding the driver motion as time-series data.

Using Machine Learning for Predicting the Best Outcomes With Electrical Muscle Stimulation for Tremors in Parkinson's Disease

Recent studies have identified that peripheral stimulation in Parkinson’s disease (PD) is effective in tremor reduction, indicating that a peripheral feedback loop plays an important role in the tremor reset mechanism. This was an open-label, quasi-experimental, pre- and post-test design, single-blind, single-group study involving 20 tremor-dominant PD patients. The objective of this study is to explore the effect of electrical muscle stimulation (EMS) as an adjunctive treatment for resting tremor during “on” period and to identify the best machine learning model to predict the suitable stimulation level that will yield the longest period of tremor reduction or tremor reset time. In this study, we used a Parkinson’s glove to evaluate, stimulate, and quantify the tremors of PD patients. This adjustable glove incorporates a 3-axis gyroscope to measure tremor signals and an EMS to provide an on-demand muscle stimulation to suppress tremors. Machine learning models were applied to identify the suitable pulse amplitude (stimulation level) in five classes that led to the longest tremor reset time. The study was registered at the www.clinicaltrials.gov under the name “The Study of Rest Tremor Suppression by Using Electrical Muscle Stimulation” (NCT02370108). Twenty tremor-dominant PD patients were recruited. After applying an average pulse amplitude of 6.25 (SD 2.84) mA and stimulation period of 440.7 (SD 560.82) seconds, the total time of tremor reduction, or tremor reset time, was 329.90 (SD 340.91) seconds. A significant reduction in tremor parameters during stimulation was demonstrated by a reduction of Unified Parkinson’s Disease Rating Scale (UPDRS) scores, and objectively, with a reduction of gyroscopic data (p < 0.05, each). None of the subjects reported any serious adverse events. We also compared gyroscopic data with five machine learning techniques: Logistic Regression, Random Forest, Support Vector Machine (SVM), Neural Network (NN), and Long-Short-Term-Memory (LSTM). The machine learning model that gave the highest accuracy was LSTM, which obtained: accuracy = 0.865 and macro-F1 = 0.736. This study confirms the efficacy of EMS in the reduction of resting tremors in PD. LSTM was identified as the most effective model for predicting pulse amplitude that would elicit the longest tremor reset time. Our study provides further insight on the tremor reset mechanism in PD.

A Comparative Study of Time Frequency Representation Techniques for Freeze of Gait Detection and Prediction

Freezing of Gait (FOG) is an impairment that affects the majority of patients in the advanced stages of Parkinson's Disease (PD). FOG can lead to sudden falls and injuries, negatively impacting the quality of life for the patients and their families. Rhythmic Auditory Stimulation (RAS) can be used to help patients recover from FOG and resume normal gait. RAS might be ineffective due to the latency between the start of a FOG event, its detection and initialization of RAS. We propose a system capable of both FOG prediction and detection using signals from tri-axial accelerometer sensors that will be useful in initializing RAS with minimal latency. We compared the performance of several time frequency analysis techniques, including moving windows extracted from the signals, handcrafted features, Recurrence Plots (RP), Short Time Fourier Transform (STFT), Discreet Wavelet Transform (DWT) and Pseudo Wigner Ville Distribution (PWVD) with Deep Learning (DL) based Long Short Term Memory (LSTM) and Convolutional Neural Networks (CNN). We also propose three Ensemble Network Architectures that combine all the time frequency representations and DL architectures. Experimental results show that our ensemble architectures significantly improve the performance compared with existing techniques. We also present the results of applying our method trained on a publicly available dataset to data collected from patients using wearable sensors in collaboration with A.T. Still University.

Towards Real-time Prediction of Freezing of Gait in Patients with Parkinsons Disease: A Novel Deep One-class Classifier

Freezing of gait (FoG) is a common motor dysfunction in individuals with Parkinsons disease. FoG impairs walking and is associated with increased fall risk. On-demand external cueing systems can detect FoG and provide stimuli to help individuals overcome freezing. Predicting FoG before onset enables preemptive cueing and may prevent FoG. However, detection and prediction remain challenging. If FoG data are not available for an individual, patient-independent models have been used to detect FoG, which have shown great sensitivity and poor specificity, or vice versa. In this study, we introduce a Deep Gait Anomaly Detector (DGAD) using a transfer learning-based approach to improve FoG detection accuracy. We also evaluate the effect of data augmentation and additional pre-FoG segments on prediction rate. Seven individuals with PD performed a series of daily walking tasks wearing inertial measurement units on their ankles. The DGAD algorithm demonstrated average sensitivity and specificity of 63.0% and 98.6% (3.2% improvement compared with the highest specificity in the literature). The target models identified 87.4% of FoG onsets, with 21.9% predicted. This study demonstrates our algorithm's potential for accurate identification of FoG and delivery of cues for patients for whom no FoG data is available for model training.

Multi-Headed Conv-LSTM Network for Heart Rate Estimation during Daily Living Activities

Non-invasive photoplethysmography (PPG) technology was developed to track heart rate during physical activity under free-living conditions. Automated analysis of PPG has made it useful in both clinical and non-clinical applications. Because of their generalization capabilities, deep learning methods can be a major direction in the search for a heart rate estimation solution based on signals from wearable devices. A novel multi-headed convolutional neural network model enriched with long short-term memory cells (MH Conv-LSTM DeepPPG) was proposed for the estimation of heart rate based on signals measured by a wrist-worn wearable device, such as PPG and acceleration signals. For the PPG-DaLiA dataset, the proposed solution improves the performance of previously proposed methods. An experimental approach was used to develop the final network architecture. The average mean absolute error (MAE) of the final solution was 6.28 bpm and Pearson’s correlation coefficient between the estimated and true heart rate values was 0.85.

Gait Parameters Measured from Wearable Sensors Reliably Detect Freezing of Gait in a Stepping in Place Task

Freezing of gait (FOG), a debilitating symptom of Parkinson’s disease (PD), can be safely studied using the stepping in place (SIP) task. However, clinical, visual identification of FOG during SIP is subjective and time consuming, and automatic FOG detection during SIP currently requires measuring the center of pressure on dual force plates. This study examines whether FOG elicited during SIP in 10 individuals with PD could be reliably detected using kinematic data measured from wearable inertial measurement unit sensors (IMUs). A general, logistic regression model (area under the curve = 0.81) determined that three gait parameters together were overall the most robust predictors of FOG during SIP: arrhythmicity, swing time coefficient of variation, and swing angular range. Participant-specific models revealed varying sets of gait parameters that best predicted FOG for each participant, highlighting variable FOG behaviors, and demonstrated equal or better performance for 6 out of the 10 participants, suggesting the opportunity for model personalization. The results of this study demonstrated that gait parameters measured from wearable IMUs reliably detected FOG during SIP, and the general and participant-specific gait parameters allude to variable FOG behaviors that could inform more personalized approaches for treatment of FOG and gait impairment in PD.

Prediction of Freezing of Gait in Parkinson's Disease Using Wearables and Machine Learning

Freezing of gait (FOG) is one of the most troublesome symptoms of Parkinson’s disease, affecting more than 50% of patients in advanced stages of the disease. Wearable technology has been widely used for its automatic detection, and some papers have been recently published in the direction of its prediction. Such predictions may be used for the administration of cues, in order to prevent the occurrence of gait freezing. The aim of the present study was to propose a wearable system able to catch the typical degradation of the walking pattern preceding FOG episodes, to achieve reliable FOG prediction using machine learning algorithms and verify whether dopaminergic therapy affects the ability of our system to detect and predict FOG. Methods: A cohort of 11 Parkinson’s disease patients receiving (on) and not receiving (off) dopaminergic therapy was equipped with two inertial sensors placed on each shin, and asked to perform a timed up and go test. We performed a step-to-step segmentation of the angular velocity signals and subsequent feature extraction from both time and frequency domains. We employed a wrapper approach for feature selection and optimized different machine learning classifiers in order to catch FOG and pre-FOG episodes. Results: The implemented FOG detection algorithm achieved excellent performance in a leave-one-subject-out validation, in patients both on and off therapy. As for pre-FOG detection, the implemented classification algorithm achieved 84.1% (85.5%) sensitivity, 85.9% (86.3%) specificity and 85.5% (86.1%) accuracy in leave-one-subject-out validation, in patients on (off) therapy. When the classification model was trained with data from patients on (off) and tested on patients off (on), we found 84.0% (56.6%) sensitivity, 88.3% (92.5%) specificity and 87.4% (86.3%) accuracy. Conclusions: Machine learning models are capable of predicting FOG before its actual occurrence with adequate accuracy. The dopaminergic therapy affects pre-FOG gait patterns, thereby influencing the algorithm’s effectiveness.

Measuring freezing of gait during daily-life: an open-source, wearable sensors approach

Background

Although a growing number of studies focus on the measurement and detection of freezing of gait (FoG) in laboratory settings, only a few studies have attempted to measure FoG during daily life with body-worn sensors. Here, we presented a novel algorithm to detect FoG in a group of people with Parkinson’s disease (PD) in the laboratory (Study I) and extended the algorithm in a second cohort of people with PD at home during daily life (Study II).

Methods

In Study I, we described of our novel FoG detection algorithm based on five inertial sensors attached to the feet, shins and lumbar region while walking in 40 participants with PD. We compared the performance of the algorithm with two expert clinical raters who scored the number of FoG episodes from video recordings of walking and turning based on duration of the episodes: very short (< 1 s), short (2–5 s), and long (> 5 s). In Study II, a different cohort of 48 people with PD (with and without FoG) wore 3 wearable sensors on their feet and lumbar region for 7 days. Our primary outcome measures for freezing were the % time spent freezing and its variability.

Results

We showed moderate to good agreement in the number of FoG episodes detected in the laboratory (Study I) between clinical raters and the algorithm (if wearable sensors were placed on the feet) for short and long FoG episodes, but not for very short FoG episodes. When extending this methodology to unsupervised home monitoring (Study II), we found that percent time spent freezing and the variability of time spent freezing differentiated between people with and without FoG (p < 0.05), and that short FoG episodes account for 69% of the total FoG episodes.

Conclusion

Our findings showed that objective measures of freezing in PD using inertial sensors on the feet in the laboratory are matching well with clinical scores. Although results found during daily life are promising, they need to be validated. Objective measures of FoG with wearable technology during community-living would be useful for managing this distressing feature of mobility disability in PD.

Body-Worn Sensors for Remote Monitoring of Parkinson's Disease Motor Symptoms: Vision, State of the Art, and Challenges Ahead

The increasing prevalence of neurodegenerative conditions such as Parkinson's disease (PD) and related mobility issues places a serious burden on healthcare systems. The COVID-19 pandemic has reinforced the urgent need for better tools to manage chronic conditions remotely, as regular access to clinics may be problematic. Digital health technology in the form of remote monitoring with body-worn sensors offers significant opportunities for transforming research and revolutionizing the clinical management of PD. Significant efforts are being invested in the development and validation of digital outcomes to support diagnosis and track motor and mobility impairments "off-line". Imagine being able to remotely assess your patient, understand how well they are functioning, evaluate the impact of any recent medication/intervention, and identify the need for urgent follow-up before overt, irreparable change takes place? This could offer new pragmatic solutions for personalized care and clinical research. So the question remains: how close are we to achieving this? Here, we describe the state-of-the-art based on representative papers published between 2017 and 2020. We focus on remote (i.e., real-world, daily-living) monitoring of PD using body-worn sensors (e.g., accelerometers, inertial measurement units) for assessing motor symptoms and their complications. Despite the tremendous potential, existing challenges exist (e.g., validity, regulatory) that are preventing the widespread clinical adoption of body-worn sensors as a digital outcome. We propose a roadmap with clear recommendations for addressing these challenges and future directions to bring us closer to the implementation and widespread adoption of this important way of improving the clinical care, evaluation, and monitoring of PD.

Inertial Sensor-Based Instrumented Cane for Real-Time Walking Cane Kinematics Estimation

Falls are among the main causes of injuries in elderly individuals. Balance and mobility impairment are major indicators of fall risk in this group. The objective of this research was to develop a fall risk feedback system that operates in real time using an inertial sensor-based instrumented cane. Based on inertial sensor data, the proposed system estimates the kinematics (contact phase and orientation) of the cane. First, the contact phase of the cane was estimated by a convolutional neural network. Next, various algorithms for the cane orientation estimation were compared and validated using an optical motion capture system. The proposed cane contact phase prediction model achieved higher accuracy than the previous models. In the cane orientation estimation, the Madgwick filter yielded the best results overall. Finally, the proposed system was able to estimate both the contact phase and orientation in real time in a single-board computer.

Using Wearable Sensors and Machine Learning to Automatically Detect Freezing of Gait during a FOG-Provoking Test

Freezing of gait (FOG) is a debilitating motor phenomenon that is common among individuals with advanced Parkinson's disease. Objective and sensitive measures are needed to better quantify FOG. The present work addresses this need by leveraging wearable devices and machine-learning methods to develop and evaluate automated detection of FOG and quantification of its severity. Seventy-one subjects with FOG completed a FOG-provoking test while wearing three wearable sensors (lower back and each ankle). Subjects were videotaped before (OFF state) and after (ON state) they took their antiparkinsonian medications. Annotations of the videos provided the "ground-truth" for FOG detection. A leave-one-patient-out validation process with a training set of 57 subjects resulted in 84.1% sensitivity, 83.4% specificity, and 85.0% accuracy for FOG detection. Similar results were seen in an independent test set (data from 14 other subjects). Two derived outcomes, percent time frozen and number of FOG episodes, were associated with self-report of FOG. Bother derived-metrics were higher in the OFF state than in the ON state and in the most challenging level of the FOG-provoking test, compared to the least challenging level. These results suggest that this automated machine-learning approach can objectively assess FOG and that its outcomes are responsive to therapeutic interventions.