Fusion of Haptic and Gesture Sensors for Rehabilitation of Bimanual Coordination and Dexterous Manipulation

Assessment of finger dexterity through the DIGITS joint tracking web application-An evaluation study with comparison to the nine-hole pegboard test

BACKGROUND

Hand dexterity is an important clinical marker after hand surgery as it can greatly impact one's ability to perform their day-to-day activities. With the increased focus on remote patient monitoring after hand surgery, new technologies are required to remotely monitor hand dexterity.

PURPOSE

The purpose of this study is to identify dexterity outcomes using the web application "DIGITS" and compare these outcomes to the nine-hole-pegboard test (NHPT).

STUDY DESIGN

Cross-sectional observational study.

METHODS

This was a two-part study with a pilot of our remote dexterity design using DIGITS followed by a validation study comparing DIGITS to a gold-standard metric of dexterity, NHPT. The pilot study recruited 42 healthy subjects between the ages of 18-65 to perform a remote finger tapping exercise using DIGITS. The second part of the study included 50 subjects between the ages of 18-65. Participants completed a demographic questionnaire and then completed three finger tapping sequences for 20 seconds using DIGITS and three trials of the NHPT with each hand. Correlational analyses were done to compare the DIGITS dexterity test with the NHPT.

RESULTS

Four outcome measures to assess dexterity were identified, which included (1) total sequences completed in 20 seconds, (2) time to complete 10 sequences, (3) average frequency per sequence, and (3) sequence accuracy. Significant negative correlations were found with the NHPT and total sequences completed in 20 seconds in both dominant and non-dominant hand trials. Additionally, significant negative correlations were found between the NHPT and the time to complete 10 sequences and average frequency in the non-dominant hand trials.

CONCLUSIONS

This study shows promising results for the use of DIGITS as a remote measure of hand dexterity. The total number of sequences completed significantly correlates with the NHPT and should be further explored in representative patient populations.

A non-contact system for intraoperative quantitative assessment of bradykinesia in deep brain stimulation surgery

BACKGROUND AND OBJECTIVE

Deep brain stimulation (DBS) is an effective treatment for a number of neurological diseases, especially for the advanced stage of Parkinson's disease (PD). Objective assessment of patients' motor symptoms is crucial for accurate electrode targeting and treatment. Existing approaches suffer from subjective variability or interference with voluntary motion. This work is aimed to establish an objective assessment system to quantify bradykinesia in DBS surgery.

METHODS

Based on the analysis of the requirements for intraoperative assessment, we developed a system with non-contact measurement, online movement feature extraction, and interactive data analysis and visualization. An optical sensor, Leap Motion Controller (LMC), was taken to detect hand movement in three clinical tasks. A graphic user interface was designed to process, compare and visualize the collected data and assessment results online. Quantified movement features include amplitude, frequency, velocity, their decrement and variability, etc. Technical validation of the system was performed with a motion capture system (Mocap), with respect to data-level and feature-level accuracy and reliability. Clinical validation was conducted with 20 PD patients for intraoperative assessments in DBS surgery. Treatment responses with respect to the bradykinesia movement features were analyzed. Single case analysis and group statistical analysis were performed to examine the differences between preoperative and intraoperative performance, and the correlation between the clinical ratings and the quantified assessment was analyzed.

RESULTS

For the movements measured by LMC and Mocap, the average Pearson's correlation coefficient was 0.986, and the mean amplitude difference was 2.11 mm. No significant difference was found for all movement features quantified by LMC and Mocap. For the clinical tests, key movement features showed significant differences between the preoperative baseline and intraoperative performance when the brain stimulation was ON. The assessment results were significantly correlated with the MDS-UPDRS clinical ratings.

CONCLUSIONS

The proposed non-contact system has established itself as an objective intraoperative assessment, analysis, and visualization tool for DBS treatment of Parkinson's disease.

A Robotic System to Deliver Multiple Physically Bimanual Tasks via Varying Force Fields

Individuals with physical limb disabilities are often restricted to perform activities of daily life (ADLs). While efficacy of bilateral training has been demonstrated in improving physical coordination of human limbs, few robots have been developed in simulating people's ADLs integrated with task-specific force field control. This study sought to develop a bilateral robot for better task rendering of general ADLs (gADLs), where gADL-consistent workspace is achieved by setting linear motors in series, and haptic rendering of multiple bimanual tasks (coupled, uncoupled and semi-coupled) is enabled by regulating force fields between robotic handles. Experiments were conducted with human users, and our results present a viable method of a single robotic system in simulating multiple physically bimanual tasks. In future, the proposed robotic system is expected to be serving as a coordination training device, and its clinical efficacy will be also investigated.

Workspace Volume of Human Bimanual Precision Manipulation Influenced by the Wrist Configuration and Finger Combination

Bimanual precision manipulation is an essential ability in daily human lives. However, the kinematic ability of bimanual precision manipulation due to its complexity and randomness was rarely discussed. This study firstly presents an objective quantitative evaluation of bimanual precision manipulation based on workspace volume. It focuses on studying the effects of the wrist and finger factors on the bimanual manipulation abilities by measuring the workspaces through which ten participants manipulated an object under the 12 situations (3 wrist configurations × 4 finger combinations). The results show that the wrists participation significantly increases the workspace for bimanual precision manipulation, while different finger combinations also substantially affect workspace volume. Therefore, we found an optimal hand situation (two indexes cooperating with the wrists participation), allowing the workspace to reach a volume of 1600cm³, which is ten times higher than the worst situation. Furthermore, the involvement of the right thumb can significantly increase the contribution ratio of finger movement in bimanual precision manipulation, making the movement more accurate and stable. The study has the potential to contribute to the researches in many domains, ranging from developing surgical devices, training doctors in microsurgical techniques, providing normative data for rehabilitation.

What the Tech? The Management of Neurological Dysfunction Through the Use of Digital Technology

Worldwide, it is estimated that millions of individuals suffer from a neurological disorder which can be the result of head injuries, ischaemic events such as a stroke, or neurodegenerative disorders such as Parkinson's disease (PD) and multiple sclerosis (MS). Problems with mobility and hemiparesis are common for these patients, making daily life, social factors and independence heavily affected. Current therapies aimed at improving such conditions are often tedious in nature, with patients often losing vital motivation and positive outlook towards their rehabilitation. The interest in the use of digital technology in neuro-rehabilitation has skyrocketed in the past decade. To gain insight, a systematic review of the literature in the field was conducting following the Preferred Reporting Items for Systematic Review and Meta-analyses (PRISMA) guidelines for three categories: stroke, Parkinson's disease and multiple sclerosis. It was found that the majority of the literature (84%) was in favour of the use of digital technologies in the management of neurological dysfunction; with some papers taking a "neutral" or "against" standpoint. It was found that the use of technologies such as virtual reality (VR), robotics, wearable sensors and telehealth was highly accepted by patients, helped to improve function, reduced anxiety and make therapy more accessible to patients living in more remote areas. The most successful therapies were those that used a combination of conventional therapies and new digital technologies.

Mirror Therapy Using Gesture Recognition for Upper Limb Function, Neck Discomfort, and Quality of Life After Chronic Stroke: A Single-Blind Randomized Controlled Trial

Background

Mirror therapy for stroke patients was reported to be effective in improving upper-extremity motor function and daily life activity performance. In addition, game-based virtual reality can be realized using a gesture recognition (GR) device, and various tasks can be presented. Therefore, this study investigated changes in upper-extremity motor function, quality of life, and neck discomfort when using a GR device for mirror therapy to observe the upper extremities reflected in the mirror.

Material/Methods

A total of 36 subjects with chronic stroke were randomly divided into 3 groups: GR mirror therapy (n=12), conventional mirror therapy (n=12), and control (n=12) groups. The GR therapy group performed 3D motion input device-based mirror therapy, the conventional mirror therapy group underwent general mirror therapy, and the control group underwent sham therapy. Each group underwent 15 (30 min/d) intervention sessions (3 d/wk for 5 weeks). All subjects were assessed by manual function test, neck discomfort score, and Short-Form 8 in pre- and post-test.

Results

Upper-extremity function, depression, and quality of life in the GR mirror therapy group were significantly better than in the control group. The changes of neck discomfort in the conventional mirror therapy and control groups were significantly greater than in the GR mirror therapy group.

Conclusions

We found that GR device-based mirror therapy is an intervention that improves upper-extremity function, neck discomfort, and quality of life in patients with chronic stroke.

The SonicHand Protocol for Rehabilitation of Hand Motor Function: A Validation and Feasibility Study

Musical sonification therapy is a new technique that can reinforce conventional rehabilitation treatments by increasing therapy intensity and engagement through challenging and motivating exercises. The aim of this paper is to evaluate the feasibility and validity of the SonicHand protocol, a new training and assessment method for the rehabilitation of hand function. The study was conducted in 15 healthy individuals and 15 stroke patients. The feasibility of implementation of the training protocol was tested in stroke patients only, who practiced a series of exercises concurrently to music sequences produced by specific movements. The assessment protocol evaluated hand motor performance during pronation/supination, wrist horizontal flexion/extension, and hand grasp without sonification. From hand position data, 15 quantitative parameters were computed evaluating mean velocity, movement smoothness, and angular excursions of hand/fingers. We validated this assessment in terms of its ability to discriminate between patients and healthy subjects, test-retest reliability and concurrent validity with the upper limb section of the Fugl-Meyer scale (FM), the functional independence measure (FIM), and the Box and Block Test (BBT). All patients showed a good understanding of the assigned tasks and were able to correctly execute the proposed training protocol, confirming its feasibility. A moderate-to-excellent intraclass correlation coefficient was found in 8/15 computed parameters. The moderate-to-strong correlation was found between the measured parameters and the clinical scales. The SonicHand training protocol is feasible and the assessment protocol showed good to excellent between-group discrimination ability, reliability, and concurrent validity, thus enabling the implementation of new personalized and motivating training programs employing sonification for the rehabilitation of hand function.

A Game Changer: 'The Use of Digital Technologies in the Management of Upper Limb Rehabilitation'

Hemiparesis is a symptom of residual weakness in half of the body, including the upper extremity, which affects the majority of post stroke survivors. Upper limb function is essential for daily life and reduction in movements can lead to tremendous decline in quality of life and independence. Current treatments, such as physiotherapy, aim to improve motor functions, however due to increasing NHS pressure, growing recognition on mental health, and close scrutiny on disease spending there is an urgent need for new approaches to be developed rapidly and sufficient resources devoted to stroke disease. Fortunately, a range of digital technologies has led to revived rehabilitation techniques in captivating and stimulating environments. To gain further insight, a meta-analysis literature search was carried out using the Preferred Reporting Items for Systematic Review and Meta-Analyses (PRISMA) method. Articles were categorized and pooled into the following groups; pro/anti/neutral for the use of digital technology. Additionally, most literature is rationalised by quantitative and qualitative findings. Findings displayed, the majority of the inclusive literature is supportive of the use of digital technologies in the rehabilitation of upper extremity following stroke. Overall, the review highlights a wide understanding and promise directed into introducing devices into a clinical setting. Analysis of all four categories; (1) Digital Technology, (2) Virtual Reality, (3) Robotics and (4) Leap Motion displayed varying qualities both-pro and negative across each device. Prevailing developments on use of these technologies highlights an evolutionary and revolutionary step into utilizing digital technologies for rehabilitation purposes due to the vast functional gains and engagement levels experienced by patients. The influx of more commercialised and accessible devices could alter stroke recovery further with initial recommendations for combination therapy utilizing conventional and digital resources.

Ambient Intelligence Environment for Home Cognitive Telerehabilitation

Higher life expectancy is increasing the number of age-related cognitive impairment cases. It is also relevant, as some authors claim, that physical exercise may be considered as an adjunctive therapy to improve cognition and memory after strokes. Thus, the integration of physical and cognitive therapies could offer potential benefits. In addition, in general these therapies are usually considered boring, so it is important to include some features that improve the motivation of patients. As a result, computer-assisted cognitive rehabilitation systems and serious games for health are more and more present. In order to achieve a continuous, efficient and sustainable rehabilitation of patients, they will have to be carried out as part of the rehabilitation in their own home. However, current home systems lack the therapist’s presence, and this leads to two major challenges for such systems. First, they need sensors and actuators that compensate for the absence of the therapist’s eyes and hands. Second, the system needs to capture and apply the therapist’s expertise. With this aim, and based on our previous proposals, we propose an ambient intelligence environment for cognitive rehabilitation at home, combining physical and cognitive activities, by implementing a Fuzzy Inference System (FIS) that gathers, as far as possible, the knowledge of a rehabilitation expert. Moreover, smart sensors and actuators will attempt to make up for the absence of the therapist. Furthermore, the proposed system will feature a remote monitoring tool, so that the therapist can supervise the patients’ exercises. Finally, an evaluation will be presented where experts in the rehabilitation field showed their satisfaction with the proposed system.

Effectiveness of Serious Games for Leap Motion on the Functionality of the Upper Limb in Parkinson's Disease: A Feasibility Study

The design and application of Serious Games (SG) based on the Leap Motion sensor are presented as a tool to support the rehabilitation therapies for upper limbs. Initially, the design principles and their implementation are described, focusing on improving both unilateral and bilateral manual dexterity and coordination. The design of the games has been supervised by specialized therapists. To assess the therapeutic effectiveness of the proposed system, a protocol of trials with Parkinson's patients has been defined. Evaluations of the physical condition of the participants in the study, at the beginning and at the end of the treatment, are carried out using standard tests. The specific measurements of each game give the therapist more detailed information about the patients' evolution after finishing the planned protocol. The obtained results support the fact that the set of developed video games can be combined to define different therapy protocols and that the information obtained is richer than the one obtained through current clinical metrics, serving as method of motor function assessment.

Quantitative assessment of paretic limb dexterity and interlimb coordination during bilateral arm rehabilitation training

In neuro-rehabilitation after stroke, the conventional constrained induced movement therapy (CIMT) has been well-accepted. Existing bilateral trainings are mostly on mirrored symmetrical motion. However, complementary bilateral movements are dominantly involved in activities of daily living (ADLs), and functional bilateral therapies may bring better skill transfer from trainings to daily life. Neurophysiological evidence is also growing. In this work, we firstly introduce our bilateral arm training system realized with a haptic interface and a motion sensor, as well as the tasks that have been designed to train both the manipulation function of the paretic arm and coordination of bilateral upper limbs. Then, we propose quantitative measures for functional assessment of complementary bilateral training performance, including kinematic behavior indices, smoothness, submovement and bimanual coordination. After that, we describe the experiments with healthy subjects and the results with respect to these quantitative measures. Feasibility and sensitivity of the proposed indices were evaluated through comparison of unilateral and bilateral training outcomes. The proposed bilateral training system and tasks, as well as the quantitative measures, have been demonstrated effective for training and assessment of unilateral and bilateral arm functions.

Serious games for upper limb rehabilitation: a systematic review

The aim of this research is to carry out a systematic review of the use of technological gaming platforms with serious games in the upper limb rehabilitation of patients with neuromotor disorders. Through a systematic review, the first two authors defined the inclusion criteria and extracted the data, resulting in 38 studies collected from B-On, PubMed and Medline. Ninety-two per cent of the selected articles were published since 2010. This review documents 35 different gaming platforms types. Twenty-one of the 38 articles included in this review conducted a clinical trial and of those only eight report improvements in the target population following the use of the games and platforms. This review concludes that a new paradigm is emerging in the rehabilitation field, characterized by the systematic use of technological gaming platforms with serious games in/for rehabilitation. The use of this approach seems to be beneficial. However, to facilitate the full integration of these platforms, it is necessary to conduct more research in this area, explore new approaches and carry out in-depth clinical studies into the benefits of these platforms. Implications for rehabilitation This review states that the use serious games and gaming platforms for upper limb rehabilitation are starting a new paradigm in the rehabilitation. For a full integration of this technologies in the rehabilitation field more studies are needed.

Leap Motion-based virtual reality training for improving motor functional recovery of upper limbs and neural reorganization in subacute stroke patients

Virtual reality is nowadays used to facilitate motor recovery in stroke patients. Most virtual reality studies have involved chronic stroke patients; however, brain plasticity remains good in acute and subacute patients. Most virtual reality systems are only applicable to the proximal upper limbs (arms) because of the limitations of their capture systems. Nevertheless, the functional recovery of an affected hand is most difficult in the case of hemiparesis rehabilitation after a stroke. The recently developed Leap Motion controller can track the fine movements of both hands and fingers. Therefore, the present study explored the effects of a Leap Motion-based virtual reality system on subacute stroke. Twenty-six subacute stroke patients were assigned to an experimental group that received virtual reality training along with conventional occupational rehabilitation, and a control group that only received conventional rehabilitation. The Wolf motor function test (WMFT) was used to assess the motor function of the affected upper limb; functional magnetic resonance imaging was used to measure the cortical activation. After four weeks of treatment, the motor functions of the affected upper limbs were significantly improved in all the patients, with the improvement in the experimental group being significantly better than in the control group. The action performance time in the WMFT significantly decreased in the experimental group. Furthermore, the activation intensity and the laterality index of the contralateral primary sensorimotor cortex increased in both the experimental and control groups. These results confirmed that Leap Motion-based virtual reality training was a promising and feasible supplementary rehabilitation intervention, could facilitate the recovery of motor functions in subacute stroke patients. The study has been registered in the Chinese Clinical Trial Registry (registration number: ChiCTR-OCH-12002238).