Intelligent Object Grasping With Sensor Fusion for Rehabilitation and Assistive Applications

Neural Correlation of EEG and Eye Movement in Natural Grasping Intention Estimation

Decoding the user's natural grasp intent enhances the application of wearable robots, improving the daily lives of individuals with disabilities. Electroencephalogram (EEG) and eye movements are two natural representations when users generate grasp intent in their minds, with current studies decoding human intent by fusing EEG and eye movement signals. However, the neural correlation between these two signals remains unclear. Thus, this paper aims to explore the consistency between EEG and eye movement in natural grasping intention estimation. Specifically, six grasp intent pairs are decoded by combining feature vectors and utilizing the optimal classifier. Extensive experimental results indicate that the coupling between the EEG and eye movements intent patterns remains intact when the user generates a natural grasp intent, and concurrently, the EEG pattern is consistent with the eye movements pattern across the task pairs. Moreover, the findings reveal a solid connection between EEG and eye movements even when taking into account cortical EEG (originating from the visual cortex or motor cortex) and the presence of a suboptimal classifier. Overall, this work uncovers the coupling correlation between EEG and eye movements and provides a reference for intention estimation.

Design, Control, and Experimental Evaluation of A Novel Robotic Glove System for Patients with Brachial Plexus Injuries

This paper presents the development of an exoskeleton glove system for people who suffer from brachial plexus injuries, aiming to assist their lost grasping functionality. The robotic system consists of a portable glove system and an embedded controller. The glove system consists of Linear Series Elastic Actuators (LSEA), Rotary Series Elastic Actuators (RSEA), and optimized finger linkages to provide imitated human motion to each finger and a coupled motion of the hand. The design principles and optimization strategies were investigated to balance functionality, portability, and stability. The model-based force control strategy compensated with a backlash model and model-free force control strategy are presented and compared. Results show that our proposed model-free control method achieves the goal of accurate force control. Finally, experiments were conducted with the prototype of the developed integrated exoskeleton glove system. Results from 3 subjects with 150 trials show that our proposed exoskeleton glove system has the potential to be used as a rehabilitation device for patients.

Development and Validation of a System for the Assessment and Recovery of Grip Force Control

The ability to finely control hand grip forces can be compromised by neuromuscular or musculoskeletal disorders. Therefore, it is recommended to include the training and assessment of grip force control in rehabilitation therapy. The benefits of robot-mediated therapy have been widely reported in the literature, and its combination with virtual reality and biofeedback can improve rehabilitation outcomes. However, the existing systems for hand rehabilitation do not allow both monitoring/training forces exerted by single fingers and providing biofeedback. This paper describes the development of a system for the assessment and recovery of grip force control. An exoskeleton for hand rehabilitation was instrumented to sense grip forces at the fingertips, and two operation modalities are proposed: (i) an active-assisted training to assist the user in reaching target force values and (ii) virtual reality games, in the form of tracking tasks, to train and assess the user's grip force control. For the active-assisted modality, the control of the exoskeleton motors allowed generating additional grip force at the fingertips, confirming the feasibility of this modality. The developed virtual reality games were positively accepted by the volunteers and allowed evaluating the performance of healthy and pathological users.

Development and Experimental Evaluation of a Novel Portable Haptic Robotic Exoskeleton Glove System for Patients with Brachial Plexus Injuries

This paper presents the development and experimental evaluation of a portable haptic exoskeleton glove system designed for people who suffer from brachial plexus injuries to restore their lost grasping functionality. The proposed glove system involves force perception, linkage-driven finger mechanism, and personalized voice control to achieve various grasping functionality requirements. The fully integrated system provides our wearable device with lightweight, portable, and comfortable characterization for grasping objects used in daily activities. Rigid articulated linkages powered by Series Elastic Actuators (SEAs) with slip detection on the fingertips provide stable and robust grasp for multiple objects. The passive abduction-adduction motion of each finger is also considered to provide better grasping flexibility for the user. The continuous voice control with bio-authentication also provides a hands-free user interface. The experiments with different objects verify the functionalities and capabilities of the proposed exoskeleton glove system in grasping objects with various shapes and weights used in activities of daily living (ADLs).

Personalized Voice Activated Grasping System for a Robotic Exoskeleton Glove ★

This paper proposes a novel human machine interface (HMI) and electronics system design to control a rehabilitation robotic exoskeleton glove. Such system can be activated with biometric authentication using the user's voice, take voice commands as input, recognize the command and perform biometric authentication in real-time with limited computing power, and execute the command on the exoskeleton. The electronics design is a stand-alone plug-and-play modulated design independent of the exoskeleton design. This personalized voice activated grasping system achieves better wearability, lower latency, and improved security than any existing exoskeleton glove control system.

Sensorimotor Rhythm-Brain Computer Interface With Audio-Cue, Motor Observation and Multisensory Feedback for Upper-Limb Stroke Rehabilitation: A Controlled Study

Several studies have shown the positive clinical effect of brain computer interface (BCI) training for stroke rehabilitation. This study investigated the efficacy of the sensorimotor rhythm (SMR)-based BCI with audio-cue, motor observation and multisensory feedback for post-stroke rehabilitation. Furthermore, we discussed the interaction between training intensity and training duration in BCI training. Twenty-four stroke patients with severe upper limb (UL) motor deficits were randomly assigned to two groups: 2-week SMR-BCI training combined with conventional treatment (BCI Group, BG, n = 12) and 2-week conventional treatment without SMR-BCI intervention (Control Group, CG, n = 12). Motor function was measured using clinical measurement scales, including Fugl-Meyer Assessment-Upper Extremities (FMA-UE; primary outcome measure), Wolf Motor Functional Test (WMFT), and Modified Barthel Index (MBI), at baseline (Week 0), post-intervention (Week 2), and follow-up week (Week 4). EEG data from patients allocated to the BG was recorded at Week 0 and Week 2 and quantified by mu suppression means event-related desynchronization (ERD) in mu rhythm (8–12 Hz). All functional assessment scores (FMA-UE, WMFT, and MBI) significantly improved at Week 2 for both groups (p < 0.05). The BG had significantly higher FMA-UE and WMFT improvement at Week 4 compared to the CG. The mu suppression of bilateral hemisphere both had a positive trend with the motor function scores at Week 2. This study proposes a new effective SMR-BCI system and demonstrates that the SMR-BCI training with audio-cue, motor observation and multisensory feedback, together with conventional therapy may promote long-lasting UL motor improvement.

Clinical Trial Registration: [http://www.chictr.org.cn], identifier [ChiCTR2000041119].

Design of Wearable Hand Rehabilitation Glove With Bionic Fiber-Reinforced Actuator

Background: The hand motor function is lost and activities in daily life (ADLs) are affected due to some illnesses such as stroke and hemiplegia. As a coping way, we present a wearable rehabilitation glove with the bionic actuator for restoring the hand function and the motor control ability lost by stroke patients. Methods: The soft pneumatic bionic actuator (SPBA) is developed on the basis of the research of human hand bone structure and finger joint characteristics and a series of tests are conducted. Besides, we built the rehabilitation glove system based on the proposed SPBAs to verify the availability due to typical gesture, mirror therapy (MT) and grasping experiment for irregular objects. Result: The bending angle of SPBA can reach 260°. The output force of it can reach 5.1N with 0.25 MPa air pressure input. The maximum variance of the bending angle can be concluded at 5.1° in MT. The grasping experiments of the glove worn on the hand or not shows the proposed glove is flexible, the grip force is large and achieve stable grasping of objects. Conclusion: The designed SPBA is satisfied with the requirements of rehabilitation training and the proposed glove restore the normal hand motion of patients in ADLs.

A Hybrid Arm-Hand Rehabilitation Robot With EMG-Based Admittance Controller

Reach-and-grasp is one of the most fundamental activities in daily life, while few rehabilitation robots provide integrated and active training of the arm and hand for patients after stroke to improve their mobility. In this study, a novel hybrid arm-hand rehabilitation robot (HAHRR) was built for the reach-and-grasp task. This hybrid structure consisted of a cable-driven module for three-dimensional arm motion and an exoskeleton for hand motion, which enabled assistance of the arm and hand simultaneously. To implement active compliance control, an EMG-based admittance controller was applied to the HAHRR. Experimental results showed that the HAHRR with the EMG-based admittance controller could not only assist the subject in fulfilling the reach-and-grasp task, but also generate smoother trajectories compared with the force-sensing-based admittance controller. These findings also suggested that the proposed approach might be applicable to post-stroke arm-hand rehabilitation training.

Stable Grasp Control With a Robotic Exoskeleton Glove

An exoskeleton robotic glove intended for patients who have suffered paralysis of the hand due to stroke or other factors has been developed and integrated. The robotic glove has the potential to aid patients with grasping objects as part of their daily life activities. Grasp stability was studied and researched by various research groups, but mainly focused on robotic grippers by devising conditions for a stable grasp of objects. Maintaining grasp stability is important so as to reduce the chances of the object slipping and dropping. But there was little focus on the grasp stability of robotic exoskeleton gloves, and most of the research was focused on mechanical design. A robotic exoskeleton glove was developed as well as novel methods to improve the grasp stability. The glove is constructed with rigidly coupled four-bar linkages attached to the finger tips. Each linkage mechanism has one-DOF (degree of freedom) and is actuated by a linear series elastic actuator (SEA). Two methods were developed to satisfy two of the conditions required for a stable grasp. These include deformation prevention of soft objects, and maintaining force and moment equilibrium of the objects being grasped. Simulations were performed to validate the performance of the proposed algorithms. A battery of experiments was performed on the integrated prototype in order to validate the performance of the algorithms developed.

Grasp Prediction Toward Naturalistic Exoskeleton Glove Control

This paper presents accurate grasp prediction algorithms that can be used for naturalistic, synergistic control of exoskeleton gloves with minimal user input. Recent research in exoskeleton systems has focused mainly on the development of novel soft or hard mechanical designs and actuation systems for rehabilitative and assistive applications. On the other hand, estimating user intent for intelligent grasp assistance is a problem that has remained largely unaddressed. As demonstrated by existing studies, the complex motions of human hand can be mapped to a latent space, thereby reducing perceived noise in individual joint angles as well as the number of variables upon which the prediction must be performed. To this extent, we present two latent space grasp prediction algorithms for intelligent exoskeleton glove control. The first presented algorithm is based on a linear regression to determine the slope and prediction horizon. The second algorithm is based on a Gaussian process trajectory matching where the trajectory of the grasping motion is probabilistically compared to existing data in order to form a prediction. Both algorithms were tested on published motion data collected from healthy subjects. In addition, the experimental validation of the algorithms was done using the RML glove (Robotics and Mechatronics Lab), which yielded similar prediction accuracy as compared to the simulation results. The proposed prediction algorithm can act as the backbone for a shifting authority controller that simultaneously amplifies the user's motion while guiding them toward their desired grasp. Preliminary work in this direction is also described in the paper, with directions for future research.

A General Purpose Robotic Hand Exoskeleton With Series Elastic Actuation

This paper describes the design and control of a novel hand exoskeleton. A subcategory of upper extremity exoskeletons, hand exoskeletons have promising applications in healthcare services, industrial workplaces, virtual reality, and military. Although much progress has been made in this field, most of the existing systems are position controlled and face several design challenges, including achieving minimal size and weight, difficulty enforcing natural grasping motions, exerting sufficient grip strength, ensuring the safety of the users hand, and maintaining overall user friendliness. To address these issues, this paper proposes a novel, slim, lightweight linkage mechanism design for a hand exoskeleton with a force control paradigm enabled via a compact series elastic actuator. A detailed design overview of the proposed mechanism is provided, along with kinematic and static analyses. To validate the overall proposed hand exoskeleton system, a fully integrated prototype is developed and tested in a series of experimental trials.