Glove-Net: Enhancing Grasp Classification with Multisensory Data and Deep Learning Approach

Grasp classification is pivotal for understanding human interactions with objects, with wide-ranging applications in robotics, prosthetics, and rehabilitation. This study introduces a novel methodology utilizing a multisensory data glove to capture intricate grasp dynamics, including finger posture bending angles and fingertip forces. Our dataset comprises data collected from 10 participants engaging in grasp trials with 24 objects using the YCB object set. We evaluate classification performance under three scenarios: utilizing grasp posture alone, utilizing grasp force alone, and combining both modalities. We propose Glove-Net, a hybrid CNN-BiLSTM architecture for classifying grasp patterns within our dataset, aiming to harness the unique advantages offered by both CNNs and BiLSTM networks. This model seamlessly integrates CNNs' spatial feature extraction capabilities with the temporal sequence learning strengths inherent in BiLSTM networks, effectively addressing the intricate dependencies present within our grasping data. Our study includes findings from an extensive ablation study aimed at optimizing model configurations and hyperparameters. We quantify and compare the classification accuracy across these scenarios: CNN achieved 88.09%, 69.38%, and 93.51% testing accuracies for posture-only, force-only, and combined data, respectively. LSTM exhibited accuracies of 86.02%, 70.52%, and 92.19% for the same scenarios. Notably, the hybrid CNN-BiLSTM proposed model demonstrated superior performance with accuracies of 90.83%, 73.12%, and 98.75% across the respective scenarios. Through rigorous numerical experimentation, our results underscore the significance of multimodal grasp classification and highlight the efficacy of the proposed hybrid Glove-Net architectures in leveraging multisensory data for precise grasp recognition. These insights advance understanding of human-machine interaction and hold promise for diverse real-world applications.

The use of wearable sensors to assess and treat the upper extremity after stroke: a scoping review

PURPOSE

To address the gap in the literature and clarify the expanding role of wearable sensor data in stroke rehabilitation, we summarized the methods for upper extremity (UE) sensor-based assessment and sensor-based treatment.

MATERIALS AND METHODS

The guideline outlined by the preferred reporting items for systematic reviews and meta-analysis extension for scoping reviews was used to complete this scoping review. Information pertaining to participant demographics, sensory information, data collection, data processing, data analysis, and study results were extracted from the studies for analysis and synthesis.

RESULTS

We included 43 articles in the final review. We organized the results into assessment and treatment categories. The included articles used wearable sensors to identify UE functional motion, categorize motor impairment/activity limitation, and quantify real-world use. Wearable sensors were also used to augment UE training by triggering sensory cues or providing instructional feedback about the affected UE.

CONCLUSIONS

Sensors have the potential to greatly expand assessment and treatment beyond traditional clinic-based approaches. This capability could support the quantification of rehabilitation dose, the nuanced assessment of impairment and activity limitation, the characterization of daily UE use patterns in real-world settings, and augment UE training adherence for home-based rehabilitation.IMPLICATIONS FOR REHABILITATIONSensor data have been used to assess UE functional motion, motor impairment/activity limitation, and real-world use.Sensor-assisted treatment approaches are emerging, and may be a promising tool to augment UE adherence in home-based rehabilitation.Wearable sensors may extend our ability to objectively assess UE motion beyond supervised clinical settings, and into home and community settings.

Use of Functional Data to Model the Trajectory of an IMU and Classify Levels of Motor Impairment for Stroke Patients

Motor impairment evaluations are key rehabilitation-related assessments for patients with stroke. Currently, such evaluations are subjective; they are based on physicians' judgements regarding the actions performed by patients. This leads to inconsistent clinical results. Many inertial sensing elements for motion detection have been designed. However, to more easily and rapidly evaluate motor impairment, we require a system that can collect data effectively to predict the degree of motor impairment. Lin et al. used data gloves equipped with an inertial measurement unit (IMU) to collect movement trajectories for motor impairment evaluations in patients with stroke. The present study used functional data analysis to model data trajectories to reduce the influence of noise from IMU data and proposed using coefficients of function as features for classifying motor impairment. To verify the appropriateness of feature construction, five classification methods were used to evaluate the extracted features in terms of the overall and sensor-specific ability to classify levels of motor impairment. The results indicated that the features derived from cubic smoothing splines could effectively reflect key data characteristics, and a support vector machine yielded relatively high overall and sensor-specific accuracy for distinguishing between levels of motion impairment in patients with stroke. Future data glove systems can contain cubic smoothing splines to extract hand function features and then classify motion impairment for appropriate rehabilitation programs to be prescribed.

Association Between Finger-to-Nose Kinematics and Upper Extremity Motor Function in Subacute Stroke: A Principal Component Analysis

Background

Kinematic analysis facilitates interpreting the extent and mechanisms of motor restoration after stroke. This study was aimed to explore the kinematic components of finger-to-nose test obtained from principal component analysis (PCA) and the associations with upper extremity (UE) motor function in subacute stroke survivors.

Methods

Thirty-seven individuals with subacute stroke and twenty healthy adults participated in the study. Six kinematic metrics during finger-to-nose task (FNT) were utilized to perform PCA. Clinical assessments for stroke participants included the Fugl-Meyer Assessment for Upper Extremity (FMA-UE), Action Research Arm Test (ARAT), and Modified Barthel Index (MBI).

Results

Three principal components (PC) accounting for 91.3% variance were included in multivariable regression models. PC1 (48.8%) was dominated by mean velocity, peak velocity, number of movement units (NMU) and normalized integrated jerk (NIJ). PC2 (31.1%) described percentage of time to peak velocity and movement time. PC3 (11.4%) profiled percentage of time to peak velocity. The variance explained by principal component regression in FMA-UE (R² = 0.71) were higher than ARAT (R² = 0.59) and MBI (R² = 0.29) for stroke individuals.

Conclusion

Kinematic components during finger-to-nose test identified by PCA are associated with UE motor function in subacute stroke. PCA reveals the intrinsic association among kinematic metrics, which may add value to UE assessment and future intervention targeted for kinematic components for stroke individuals.

Clinical Trial Registration

Chinese Clinical Trial Registry (http://www.chictr.org.cn/) on 17 October 2019, identifier: ChiCTR1900026656.

Smart Glove Integrated with Tunable MWNTs/PDMS Fibers Made of a One-Step Extrusion Method for Finger Dexterity, Gesture, and Temperature Recognition

Flexible wearable devices have proven to be emerging tools for motion monitoring, personal healthcare, and rehabilitation training. The development of a multifunctional, flexible sensor and the integration of sensors and a smart chip for signal reading and transmission play a critical role in building a smart wearable device. In this work, a smart glove based on multiwalled carbon nanotubes/poly(dimethylsiloxane) (MWNTs/PDMS) fibers is developed for gesture and temperature recognition. First, the well-tunable, stretchable, and thermal-sensitive MWNTs/PDMS fibers are fabricated via a facile and cost-effective one-step extrusion method. The obtained fibers exhibit an outstanding linear relationship between resistance change and strain in the range of 0-120% and excellent cyclic stability and durability after 20 000 cycles of 50% tension. They also present a linear relationship of resistance change and temperature of 0.55% °C^-1 with a correlation coefficient of 0.998 in the range of 0-100 °C. The fibers, as parts of wearable sensors, are then integrated into a smart glove along with a custom-made data acquisition chip to recognize finger dexterity, gestures, and temperature signals and output them through a screen display, an audio system, and Bluetooth transmission. The highly integrated, low-cost, and multifunctional glove holds great potential for various applications, such as sign language recognition, rehabilitation training, and telemedicine in the Internet-of-Things era.