A multimodal screening system for elderly neurological diseases based on deep learning

In this paper, we propose a deep-learning-based algorithm for screening neurological diseases. We proposed various examination protocols for screening neurological diseases and collected data by video-recording persons performing these protocols. We converted video data into human landmarks that capture action information with a much smaller data dimension. We also used voice data which are also effective indicators of neurological disorders. We designed a subnetwork for each protocol to extract features from landmarks or voice and a feature aggregator that combines all the information extracted from the protocols to make a final decision. Multitask learning was applied to screen two neurological diseases. To capture meaningful information about these human landmarks and voices, we applied various pre-trained models to extract preliminary features. The spatiotemporal characteristics of landmarks are extracted using a pre-trained graph neural network, and voice features are extracted using a pre-trained time-delay neural network. These extracted high-level features are then passed onto the subnetworks and an additional feature aggregator that are simultaneously trained. We also used various data augmentation techniques to overcome the shortage of data. Using a frame-length staticizer that considers the characteristics of the data, we can capture momentary tremors without wasting information. Finally, we examine the effectiveness of different protocols and different modalities (different body parts and voice) through extensive experiments. The proposed method achieves AUC scores of 0.802 for stroke and 0.780 for Parkinson's disease, which is effective for a screening system.

Upper extremity asymmetry due to nerve injuries or central neurologic conditions: a scoping review

BACKGROUND

Peripheral nerve injuries and central neurologic conditions can result in extensive disabilities. In cases with unilateral impairment, assessing the asymmetry between the upper extremity has been used to assess outcomes of treatment and severity of injury. A wide variety of validated and novel tests and sensors have been utilized to determine the upper extremity asymmetry. The purpose of this article is to review the literature and define the current state of the art for describing upper extremity asymmetry in patients with peripheral nerve injuries or central neurologic conditions.

METHOD

An electronic literature search of PubMed, Scopus, Web of Science, OVID was performed for publications between 2000 to 2022. Eligibility criteria were subjects with neurological conditions/injuries who were analyzed for dissimilarities in use between the upper extremities. Data related to study population, target condition/injury, types of tests performed, sensors used, real-world data collection, outcome measures of interest, and results of the study were extracted. Sackett's Level of Evidence was used to judge the quality of the articles.

RESULTS

Of the 7281 unique articles, 112 articles met the inclusion criteria for the review. Eight target conditions/injuries were identified (Brachial Plexus Injury, Cerebral Palsy, Multiple Sclerosis, Parkinson's Disease, Peripheral Nerve Injury, Spinal Cord Injury, Schizophrenia, and stroke). The tests performed were classified into thirteen categories based on the nature of the test and data collected. The general results related to upper extremity asymmetry were listed for all the reviewed articles. Stroke was the most studied condition, followed by cerebral palsy, with kinematics and strength measurement tests being the most frequently used tests. Studies with a level of evidence level II and III increased between 2000 and 2021. The use of real-world evidence-based data, and objective data collection tests also increased in the same period.

CONCLUSION

Adequately powered randomized controlled trials should be used to study upper extremity asymmetry. Neurological conditions other than stroke should be studied further. Upper extremity asymmetry should be measured using objective outcome measures like motion tracking and activity monitoring in the patient's daily living environment.

The use of wearable sensors to assess and treat the upper extremity after stroke: a scoping review

PURPOSE

To address the gap in the literature and clarify the expanding role of wearable sensor data in stroke rehabilitation, we summarized the methods for upper extremity (UE) sensor-based assessment and sensor-based treatment.

MATERIALS AND METHODS

The guideline outlined by the preferred reporting items for systematic reviews and meta-analysis extension for scoping reviews was used to complete this scoping review. Information pertaining to participant demographics, sensory information, data collection, data processing, data analysis, and study results were extracted from the studies for analysis and synthesis.

RESULTS

We included 43 articles in the final review. We organized the results into assessment and treatment categories. The included articles used wearable sensors to identify UE functional motion, categorize motor impairment/activity limitation, and quantify real-world use. Wearable sensors were also used to augment UE training by triggering sensory cues or providing instructional feedback about the affected UE.

CONCLUSIONS

Sensors have the potential to greatly expand assessment and treatment beyond traditional clinic-based approaches. This capability could support the quantification of rehabilitation dose, the nuanced assessment of impairment and activity limitation, the characterization of daily UE use patterns in real-world settings, and augment UE training adherence for home-based rehabilitation.IMPLICATIONS FOR REHABILITATIONSensor data have been used to assess UE functional motion, motor impairment/activity limitation, and real-world use.Sensor-assisted treatment approaches are emerging, and may be a promising tool to augment UE adherence in home-based rehabilitation.Wearable sensors may extend our ability to objectively assess UE motion beyond supervised clinical settings, and into home and community settings.

Scalable Cluster Tendency Assessment for Streaming Activity Data using Recurring Shapelets

Automatic interpretation of cluster structure in rapidly arriving data streams is essential for timely detection of interesting events. Human activities often contain bursts of repeating patterns. In this paper, we propose a new relative of the Visual Assessment of Cluster Tendency (VAT) model, to interpret cluster evolution in streaming activity data where shapes of recurring patterns are important. Existing VAT algorithms are either suitable only for small batch data and unscalable to rapidly evolving streams, or cannot capture shape patterns. Our proposed incremental algorithm processes streaming data in chunks and identifies repeating patterns or shapelets from each chunk, creating a Dictionary-of-Shapes (DoS) that is updated on the fly. Each chunk is transformed into a lower dimensional representation based on it's distance from the shapelets in the current DoS. Then a small set of transformed chunks are sampled using an intelligent Maximin Random Sampling (MMRS) scheme, to create a scalable VAT image that is incrementally updated as the data stream progresses. Experiments on two upper limb activity datasets demonstrate that the proposed method can successfully and efficiently visualize clusters in long streams of data and can also identify anomalous movements.

Novel Measures of Similarity and Asymmetry in Upper Limb Activities for Identifying Hemiparetic Severity in Stroke Survivors

Stroke survivors are often characterized by hemiparesis, i.e., paralysis in one half of the body, severely affecting upper limb movements. Monitoring the progression of hemiparesis requires manual observation of limb movements at regular intervals, and hence is a labour intensive process. In this work, we use wrist-worn accelerometers for automated assessment of hemiparesis in acute stroke. We propose novel measures of similarity and asymmetry in hand activities through bivariate Poincaré analysis between two-hand accelerometer data for quantifying hemiparetic severity. The proposed descriptors characterize the distribution of activity surrogates derived from acceleration of the two hands, on a 2D bivariate Poincaré Plot. Experiments show that while the descriptors CSD1 and CSD2 can identify hemiparetic patients from control subjects, their normalized difference CSDR and the descriptors Complex Cross-Correlation Measure ( C3M) and Activity Asymmetry Index ( AAI) can distinguish between mild, moderate and severe hemiparesis. These measures are compared with traditional measures of cross-correlation and evaluated against the National Institutes of Health Stroke Scale (NIHSS), the clinical gold standard for hemiparetic severity estimation. This study, undertaken on 40 acute stroke patients with varying levels of hemiparesis and 15 healthy controls, validates the use of short length ( 5 minutes) wearable accelerometry data for identifying hemiparesis with greater clinical sensitivity. Results show that the proposed descriptors with a hierarchical classification model outperform state-of-the-art methods with overall accuracy of 0.78 and 0.85 for 4-class and 3-class hemiparesis identification respectively.

Upper limb movement profiles during spontaneous motion in acute stroke

Objective.The clinical assessment of upper limb hemiparesis in acute stroke involves repeated manual examination of hand movements during instructed tasks. This process is labour-intensive and prone to human error as well as being strenuous for the patient. Wearable motion sensors can automate the process by measuring characteristics of hand activity. Existing work in this direction either uses multiple sensors or complex instructed movements, or analyzes only thequantityof upper limb motion. These methods are obtrusive and strenuous for acute stroke patients and are also sensitive to noise. In this work, we propose to use only two wrist-worn accelerometer sensors to study thequalityof completely spontaneous upper limb motion and investigate correlation with clinical scores for acute stroke care.Approach.The velocity time series estimated from acquired acceleration data during spontaneous motion is decomposed into smaller movement elements. Measures of density, duration and smoothness of these component elements are extracted and their disparity is studied across the two hands.Main results.Spontaneous upper limb motion in acute stroke can be decomposed into movement elements that resemble point-to-point reaching tasks. These elements are smoother and sparser in the normal hand than in the hemiparetic hand, and the amount of smoothness correlates with hemiparetic severity. Features characterizing the disparity of these movement elements between the two hands show statistical significance in differentiating mild-to-moderate and severe hemiparesis. Using data from 67 acute stroke patients, the proposed method can classify the two levels of hemiparetic severity with 85% accuracy. Additionally, compared to activity-based features, the proposed method is robust to the presence of noise in acquired data.Significance.This work demonstrates that the quality of upper limb motion can characterize and identify hemiparesis in stroke survivors. This is clinically significant towards the continuous automated assessment of hemiparesis in acute stroke using minimally intrusive wearable sensors.

Analyzing Distance Measures for Upper Limb Activity Measurement in Hemiparetic Stroke Patients

Stroke survivors are often characterized by upper limb hemiparesis due to which activities in one of the hands is significantly restricted. Manual evaluation of the progression of hemiparesis in acute stroke patients involves 24x7 medical supervision, which is prone to inter-rater variability, is labor-intensive and consequently expensive in public hospitals. In this paper, we investigate the use of wrist-worn accelerometers for automated identification of upper limb hemiparesis in acute stroke. We propose a set of spontaneous and instructed movements in order to estimate two-hand activity correlation using accelerometry data. We use this information to determine the weak hand and further investigate an Activity Based Distance (ABD) measure to quantify this correlation. We compare ABD with standard time-series distance measures such as Lp norms and Dynamic Time Warping (DTW) for hemiparetic severity estimation. We study these distance measures with respect to the National Institutes of Health Stroke Scale (NIHSS), the clinical gold standard to determine hemiparetic severity, and demonstrate their suitability for developing a wearable based automated hemiparesis detection and monitoring system.Clinical relevance-This study presents a novel experimental paradigm for identifying upper limb hemiparesis in acute stroke patients using measures of two-hand activity correlation.