Markerless motion capture and measurement of hand kinematics: validation and application to home-based upper limb rehabilitation

Stretchable glove for accurate and robust hand pose reconstruction based on comprehensive motion data

We propose a compact wearable glove capable of estimating both the finger bone lengths and the joint angles of the wearer with a simple stretch-based sensing mechanism. The soft sensing glove is designed to easily stretch and to be one-size-fits-all, both measuring the size of the hand and estimating the finger joint motions of the thumb, index, and middle fingers. The system was calibrated and evaluated using comprehensive hand motion data that reflect the extensive range of natural human hand motions and various anatomical structures. The data were collected with a custom motion-capture setup and transformed into the joint angles through our post-processing method. The glove system is capable of reconstructing arbitrary and even unconventional hand poses with accuracy and robustness, confirmed by evaluations on the estimation of bone lengths (mean error: 2.1 mm), joint angles (mean error: 4.16°), and fingertip positions (mean 3D error: 4.02 mm), and on overall hand pose reconstructions in various applications. The proposed glove allows us to take advantage of the dexterity of the human hand with potential applications, including but not limited to teleoperation of anthropomorphic robot hands or surgical robots, virtual and augmented reality, and collection of human motion data.

Feasibility of using low-cost markerless motion capture for assessing functional outcomes after lower extremity musculoskeletal cancer surgery

BACKGROUND

Physical limitations are frequent and debilitating after sarcoma treatment. Markerless motion capture (MMC) could measure these limitations. Historically expensive cumbersome systems have posed barriers to clinical translation.

RESEARCH QUESTION

Can inexpensive MMC [using Microsoft KinectTM] assess functional outcomes after sarcoma surgery, discriminate between tumour sub-groups and agree with existing assessments?

METHODS

Walking, unilateral stance and kneeling were measured in a cross-sectional study of patients with lower extremity sarcomas using MMC and standard video. Summary measures of temporal, balance, gait and movement velocity were derived. Feasibility and early indicators of validity of MMC were explored by comparing MMC measures i) between tumour sub-groups; ii) against video and iii) with established sarcoma tools [Toronto Extremity Salvage Score (TESS)), Musculoskeletal Tumour Rating System (MSTS), Quality of life-cancer survivors (QoL-CS)]. Statistical analysis was conducted using SPSS v19. Tumour sub-groups were compared using Mann-Whitney U tests, MMC was compared to existing sarcoma measures using correlations and with video using Intraclass correlation coefficient agreement.

RESULTS

Thirty-four adults of mean age 43 (minimum value-maximum value 19-89) years with musculoskeletal tumours in the femur (19), pelvis/hip (3), tibia (9), or ankle/foot (3) participated; 27 had limb sparing surgery and 7 amputation. MMC was well-tolerated and feasible to deliver. MMC discriminated between surgery groups for balance (p<0.05*), agreed with video for kneeling times [ICC = 0.742; p = 0.001*] and showed moderate relationships between MSTS and gait (p = 0.022*, r = -0.416); TESS and temporal outcomes (p = 0.016* and r = -0.0557*), movement velocity (p = 0.021*, r = -0.541); QoL-CS and balance (p = 0.027*, r = 0.441) [* = statistical significance]. As MMC uncovered important relationships between outcomes, it gave an insight into how functional impairments, balance, gait, disabilities and quality of life (QoL) are associated with each other. This gives an insight into mechanisms of poor outcomes, producing clinically useful data i.e. data which can inform clinical practice and guide the delivery of targeted rehabilitation. For example, patients presenting with poor balance in various activities can be prescribed with balance rehabilitation and those with difficulty in movements or activity transitions can be managed with exercises and training to improve the quality and efficiency of the movement.

SIGNIFICANCE

In this first study world-wide, investigating the use of MMC after sarcoma surgery, MMC was found to be acceptable and feasible to assess functional outcomes in this cancer population. MMC demonstrated early indicators of validity and also provided new knowledge that functional impairments are related to balance during unilateral stance and kneeling, gait and movement velocity during kneeling and these outcomes in turn are related to disabilities and QoL. This highlighted important relationships between different functional outcomes and QoL, providing valuable information for delivering personalised rehabilitation. After completing future validation work in a larger study, this approach can offer promise in clinical settings. Low-cost MMC shows promise in assessing patient's impairments in the hospitals or their homes and guiding clinical management and targeted rehabilitation based on novel MMC outcomes affected, therefore providing an opportunity for delivering personalised exercise programmes and physiotherapy care delivery for this rare cancer.

Reliability, accuracy, and minimal detectable difference of a mixed concept marker set for finger kinematic evaluation

The study of finger biomechanics requires special tools for accurately recording finger joint data. A marker set to evaluate finger postures during activities of daily living is needed to understand finger biomechanics in order to improve prosthesis design and clinical interventions. The purpose of this study was to evaluate the reliability of a proposed hand marker set (the Warwick marker set) to capture finger kinematics using motion capture. The marker set consisted of the application of two and three marker clusters to the fingers of twelve participants who participated in the tests across two sessions. Calibration markers were applied using a custom palpation technique. Each participant performed a series of range of motion movements and held a set of objects. Intra and inter-session reliability was calculated as well as Standard Error of Measurement (SEM) and Minimal Detectable Difference (MDD). The findings showed varying levels of intra- and inter-session reliability, ranging from poor to excellent. The SEM and MDD values were lower for the intra-session range of motion and grasp evaluation. The reduced reliability can potentially be attributed to skin artifacts, differences in marker placement, and the inherent kinematic variability of finger motion. The proposed marker set shows potential to assess finger postures and analyse activities of daily living, primarily within the context of single session tests.

Accuracy and feasibility of a novel fine hand motor skill assessment using computer vision object tracking

We developed a computer vision-based three-dimension (3D) motion capture system employing two action cameras to examine fine hand motor skill by tracking an object manipulated by a hand. This study aimed to examine the accuracy and feasibility of this approach for detecting changes in a fine hand motor skill. We conducted three distinct experiments to assess the system's accuracy and feasibility. We employed two high-resolution, high-frame-rate action cameras. We evaluated the accuracy of our system in calculating the 3D locations of moving object in various directions. We also examined the system's feasibility in identifying improvement in fine hand motor skill after practice in eleven non-disabled young adults. We utilized color-based object detection and tracking to estimate the object's 3D location, and then we computed the object's kinematics, representing the endpoint goal-directed arm reaching movement. Compared to ground truth measurements, the findings demonstrated that our system can adequately estimate the 3D locations of a moving object. We also showed that the system can be used to measure the endpoint kinematics of goal-directed arm reaching movements to detect changes in fine hand motor skill after practice. Future research is needed to confirm the system's reliability and validity in assessing fine hand motor skills in patient populations.

A hand motion capture method based on infrared thermography for measuring fine motor skills in biomedicine

Many biomedical applications require fine motor skill assessments; however, real-time and contactless fine motor skill assessments are not typically implemented. In this study, we followed the 2D-to-3D pipeline principle and proposed a transformer-based spatial-temporal network to accurately regress 3D hand joint locations by inputting infrared thermal video for eliminating need of multiple cameras or RGB-D devices. We also developed a dataset composed of infrared thermal videos and ground truth annotations for training. The label represents a set of 3D joint locations from infrared optical trackers, which is considered the gold standard for clinical applications. To demonstrate their potential, the proposed method was used to measure the finger motion angle, and we investigated its accuracy by comparing the proposal with the Azure Kinect system and Leap Motion system. On the proposed dataset, the proposed method achieved a 3D hand pose mean error of less than 14 mm and outperforms the other deep learning methods. When the error thresholds were larger than approximately 35 mm, our method first to achieved excellent performance (>80%) in terms of the fraction of good frames. For the finger motion angle calculation task, the proposed and commercial systems had comparable inter-system reliability (ICC_2,1 ranging from 0.81 to 0.83) and excellent validity (Pearson's r-values ranging from 0.82 to 0.86). We believe that the proposed approaches can capture hand motion and measure finger motion angles and can be used in different biomedicine scenarios as an effective evaluation tool for fine motor skills.

Validation of two-dimensional video-based inference of finger kinematics with pose estimation

Accurate capture finger of movements for biomechanical assessments has typically been achieved within laboratory environments through the use of physical markers attached to a participant’s hands. However, such requirements can narrow the broader adoption of movement tracking for kinematic assessment outside these laboratory settings, such as in the home. Thus, there is the need for markerless hand motion capture techniques that are easy to use and accurate enough to evaluate the complex movements of the human hand. Several recent studies have validated lower-limb kinematics obtained with a marker-free technique, OpenPose. This investigation examines the accuracy of OpenPose, when applied to images from single RGB cameras, against a ‘gold standard’ marker-based optical motion capture system that is commonly used for hand kinematics estimation. Participants completed four single-handed activities with right and left hands, including hand abduction and adduction, radial walking, metacarpophalangeal (MCP) joint flexion, and thumb opposition. The accuracy of finger kinematics was assessed using the root mean square error. Mean total active flexion was compared using the Bland–Altman approach, and the coefficient of determination of linear regression. Results showed good agreement for abduction and adduction and thumb opposition activities. Lower agreement between the two methods was observed for radial walking (mean difference between the methods of 5.03°) and MCP flexion (mean difference of 6.82°) activities, due to occlusion. This investigation demonstrated that OpenPose, applied to videos captured with monocular cameras, can be used for markerless motion capture for finger tracking with an error below 11° and on the order of that which is accepted clinically.

Research on Monocular-Vision-Based Finger-Joint-Angle-Measurement System

The quantitative measurement of finger-joint range of motion plays an important role in assessing the level of hand disability and intervening in the treatment of patients. An industrial monocular-vision-based knuckle-joint-activity-measurement system is proposed with short measurement time and the simultaneous measurement of multiple joints. In terms of hardware, the system can adjust the light-irradiation angle and the light-irradiation intensity of the marker by actively adjusting the height of the light source to enhance the difference between the marker and the background and reduce the difficulty of segmenting the target marker and the background. In terms of algorithms, a combination of multiple-vision algorithms is used to compare the image-threshold segmentation and Hough outer- and inner linear detection as the knuckle-activity-range detection method of the system. To verify the accuracy of the visual-detection method, nine healthy volunteers were recruited for experimental validation, and the experimental results showed that the average angular deviation in the flexion/extension of the knuckle was 0.43° at the minimum and 0.59° at the maximum, and the average angular deviation in the adduction/abduction of the knuckle was 0.30° at the minimum and 0.81° at the maximum, which were all less than 1°. In the multi-angle velocimetry experiment, the time taken by the system was much less than that taken by the conventional method.

A system for automated acquisition of digital flexion using a 3-D camera and custom gantry

Introduction

Automated measurement of digital range of motion (ROM) may improve the accuracy of reporting and increase clinical efficiency. We hypothesize that a 3-D camera on a custom gantry will produce ROM measurements similar to those obtained with a manual goniometer.

Methods

A 3-D camera mounted on a custom gantry, was mechanized to rotate 200° around a platform. The video was processed to segment each digit and calculate joint angles in people with no history of any hand conditions or surgery to validate the system. A second-generation prototype was then assessed in people with different hand conditions. Metacarpophalangeal (MCP) and proximal interphalangeal (PIP) joint flexion were measured repeatedly with a goniometer and the automated system. The average difference between manual and automatic measurements was calculated along with intraclass correlation coefficients (ICC).

Results

In the initial validation, 1,488 manual and 1,488 automated joint measurements were obtained and the measurement algorithm was refined. In people with hand conditions, 688 manual and 688 automated joint measurements were compared. Average acquisition time was 7 s per hand, with an additional 2-3 s required for data processing. ICC between manual and automated data in the clinical study ranged from 0.65 to 0.85 for the MCP joints, and 0.22 to 0.66 for PIP joints.

Discussion

The automated system resulted in rapid data acquisition, with reliability varying by type of joint and location. It has the potential to improve efficiency in the collection of physical exam findings. Further developments of the system are needed to measure thumb and distal phalangeal motions.

Comparison of Motion Analysis Systems in Tracking Upper Body Movement of Myoelectric Bypass Prosthesis Users

Current literature lacks a comparative analysis of different motion capture systems for tracking upper limb (UL) movement as individuals perform standard tasks. To better understand the performance of various motion capture systems in quantifying UL movement in the prosthesis user population, this study compares joint angles derived from three systems that vary in cost and motion capture mechanisms: a marker-based system (Vicon), an inertial measurement unit system (Xsens), and a markerless system (Kinect). Ten healthy participants (5F/5M; 29.6 ± 7.1 years) were trained with a TouchBionic i-Limb Ultra myoelectric terminal device mounted on a bypass prosthetic device. Participants were simultaneously recorded with all systems as they performed standardized tasks. Root mean square error and bias values for degrees of freedom in the right elbow, shoulder, neck, and torso were calculated. The IMU system yielded more accurate kinematics for shoulder, neck, and torso angles while the markerless system performed better for the elbow angles. By evaluating the ability of each system to capture kinematic changes of simulated upper limb prosthesis users during a variety of standardized tasks, this study provides insight into the advantages and limitations of using different motion capture technologies for upper limb functional assessment.

Vogtareuth Rehab Depth Datasets: Benchmark for Marker-less Posture Estimation in Rehabilitation

Posture estimation using a single depth camera has become a useful tool for analyzing movements in rehabilitation. Recent advances in posture estimation in computer vision research have been possible due to the availability of large-scale pose datasets. However, the complex postures involved in rehabilitation exercises are not represented in the existing benchmark depth datasets. To address this limitation, we propose two rehabilitation-specific pose datasets containing depth images and 2D pose information of patients, both adult and children, performing rehab exercises. We use a state-of-the-art marker-less posture estimation model which is trained on a non-rehab benchmark dataset. We evaluate it on our rehab datasets, and observe that the performance degrades significantly from non-rehab to rehab, highlighting the need for these datasets. We show that our dataset can be used to train pose models to detect rehab-specific complex postures. The datasets will be released for the benefit of the research community.

Development of the Home based Virtual Rehabilitation System (HoVRS) to remotely deliver an intense and customized upper extremity training

Background

After stroke, sustained hand rehabilitation training is required for continuous improvement and maintenance of distal function.

Methods

In this paper, we present a system designed and implemented in our lab: the Home based Virtual Rehabilitation System (HoVRS). Fifteen subjects with chronic stroke were recruited to test the feasibility of the system as well as to refine the design and training protocol to prepare for a future efficacy study. HoVRS was placed in subjects’ homes, and subjects were asked to use the system at least 15 min every weekday for 3 months (12 weeks) with limited technical support and remote clinical monitoring.

Results

All subjects completed the study without any adverse events. Subjects on average spent 13.5 h using the system. Clinical and kinematic data were collected pre and post study in the subject’s home. Subjects demonstrated a mean increase of 5.2 (SEM = 0.69) on the Upper Extremity Fugl-Meyer Assessment (UEFMA). They also demonstrated improvements in six measurements of hand kinematics. In addition, a combination of these kinematic measures was able to predict a substantial portion of the variability in the subjects’ UEFMA score.

Conclusion

Persons with chronic stroke were able to use the system safely and productively with minimal supervision resulting in measurable improvements in upper extremity function.

Camera-based Hand Tracking using a Mirror-based Multi-view Setup

Current clinical practice of measuring hand joint range of motion relies on a goniometer as it is inexpensive, portable, and easy to use, but it can only measure the static angle of a single joint at a time. To measure dynamic hand motion, a camera-based system that can perform markerless hand pose estimation is attractive, as the system is ubiquitous, low-cost, and non-contact. However, camera-based systems require line-of-sight, and tracking accuracy degrades when the joint is occluded from the camera view. Thus, we propose a multi-view setup using a readily available color camera from a single mobile phone, and plane mirrors to create multiple views of the hand. This setup eliminates the complexity of synchronizing multiple cameras and reduce the issue of occlusion. Experimental results show that the multi-view setup could help to reduce the error in measuring the flexion angle of finger joints. Dynamic hand pose estimation with object interaction is also demonstrated.

Design, analysis and performance evaluation of a hand gesture platform for navigation

Prevailing technological solutions that address the problems that are experienced by the infirm and elderly people in terms of locomotion needs, offer limited options when it comes to control mechanism and customization. For more than a decade, joysticks have served the purpose of steering and navigation of autonomous wheelchairs. An alternative gesture-based method for navigation of wheelchairs by the physically impaired could very well replace the conventional joystick controls. A prototype system, 'Mudra' (Sanskrit word for gesture), incorporates a gesture capture module, developed for recognition and identification of hand gestures. Mudra is a no-nonsense user-friendly system that enables control of the navigational platform, merely by four gestures of the human hand. This paper presents a comprehensive report on the implementation of the Mudra system's hardware and software, performance analysis and safety evaluation. Verification tests of the conceptual design show promising results, where 97.8% of the hand gestures were recognized accurately. Furthermore, the response timings of wheelchairs with Mudra controls were noticeably faster than the joystick-based wheelchairs, as affirmed by live testing with wheelchair-users. Pursuant to the positive feedback from the wheelchair-user experience, we conclude that Mudra's gesture controlled wheelchairs would be a preferable alternative to joystick-controlled wheelchairs.

Classification of Therapeutic Hand Poses Using Convolutional Neural Networks

Measurement of finger active range of motion (ARoM) is essential to quantify outcomes accurately after hand surgery and during rehabilitation. Currently, finger ARoM is measured by a hand-held goniometer, which introduces measurement error. Moreover, this method is time-consuming. To speed up and simplify this process, we developed a system to measure the ARoM automatically. However, to assess the ARoM for all joints, different hand poses are required. The goal, then, is to design a classifier that achieves accurate and automatic discovery of the hand pose. According to the detected pose, the system will apply the appropriate algorithm to measure the ARoM for all fingers. Furthermore, this will enable a camera capture control system to provide the best view by moving the camera as required by each algorithm. A critical part of the system is the classifier because it controls the accuracy and compute time of the measurement. In this paper, we describe a study of different classifiers for hand pose and include results. The best classifier achieves 99% accuracy in classifying 400 test samples from five previously unseen human subjects with a compute time of 8ms per sample.

Towards Finger Motion Capture System Using FBG Sensors

This paper introduces a novel finger motion capture system using FBG (fiber Bragg grating) optical sensors. We develop two types of sensors to seamlessly reconstruct finger motion from strains induced in the FBGs. First, the shape sensor incorporates three optical fibers with multiple FBGs to reconstruct the position and orientation of a finger joint in 3D. In addition, the angle sensor is designed to measure the high curvature of bending on the finger joints. By deploying the two types of sensors on the fingers, we can reconstruct various finger motion in real time without drift over time. The accuracies of the fabricated FBG sensors are evaluated, resulting in an average error of 1.49 mm for the shape sensor at the distal tip (1.9% for the full length of the sensor) and 0.21° error for the angle sensor. We finally demonstrate finger motion tracking with the FBG sensors in real time, while measuring the multi-DOF motion at the carpometacarpal joint of the thumb and also the high curvatures of bending motion at the metacarpophalangeal and interphalangeal joints of the thumb and the index finger.

Kinect-based assessment of proximal arm non-use after a stroke

Background

After a stroke, during seated reaching with their paretic upper limb, many patients spontaneously replace the use of their arm by trunk compensation movements, even though they are able to use their arm when forced to do so. We previously quantified this proximal arm non-use (PANU) with a motion capture system (Zebris, CMS20s). The aim of this study was to validate a low-cost Microsoft Kinect-based system against the CMS20s reference system to diagnose PANU.

Methods

In 19 hemiparetic stroke individuals, the PANU score, reach length, trunk length, and proximal arm use (PAU) were measured during seated reaching simultaneously by the Kinect (v2) and the CMS20s over two testing sessions separated by two hours.

Results

Intraclass correlation coefficients (ICC) and linear regression analysis showed that the PANU score (ICC = 0.96, r² = 0.92), reach length (ICC = 0.81, r² = 0.68), trunk length (ICC = 0.97, r² = 0.94) and PAU (ICC = 0.97, r² = 0.94) measured using the Kinect were strongly related to those measured using the CMS20s. The PANU scores showed good test-retest reliability for both the Kinect (ICC = 0.76) and CMS20s (ICC = 0.72). Bland and Altman plots showed slightly reduced PANU scores in the re-test session for both systems (Kinect: − 4.25 ± 6.76; CMS20s: − 4.71 ± 7.88), which suggests a practice effect.

Conclusion

We showed that the Kinect could accurately and reliably assess PANU, reach length, trunk length and PAU during seated reaching in post stroke individuals. We conclude that the Kinect can offer a low-cost and widely available solution to clinically assess PANU for individualised rehabilitation and to monitor the progress of paretic arm recovery.

Trial registration

The study was approved by The Ethics Committee of Montpellier, France (N°ID-RCB: 2014-A00395–42) and registered in Clinical Trial (N° NCT02326688, Registered on 15 December 2014, https://clinicaltrials.gov/ct2/show/results/NCT02326688).

Design of an Inertial-Sensor-Based Data Glove for Hand Function Evaluation

Capturing hand motions for hand function evaluations is essential in the medical field. Various data gloves have been developed for rehabilitation and manual dexterity assessments. This study proposed a modular data glove with 9-axis inertial measurement units (IMUs) to obtain static and dynamic parameters during hand function evaluation. A sensor fusion algorithm is used to calculate the range of motion of joints. The data glove is designed to have low cost, easy wearability, and high reliability. Owing to the modular design, the IMU board is independent and extensible and can be used with various microcontrollers to realize more medical applications. This design greatly enhances the stability and maintainability of the glove.

Reliability and validity of the Microsoft Kinect for assessment of manual wheelchair propulsion

Concurrent validity and test-retest reliability of the Microsoft Kinect in quantification of manual wheelchair propulsion were examined. Data were collected from five manual wheelchair users on a roller system. Three Kinect sensors were used to assess test-retest reliability with a still pose. Three systems were used to assess concurrent validity of the Kinect to measure propulsion kinematics (joint angles, push loop characteristics): Kinect, Motion Analysis, and Dartfish ProSuite (Dartfish joint angles were limited to shoulder and elbow flexion). Intraclass correlation coefficients revealed good reliability (0.87-0.99) between five of the six joint angles (neck flexion, shoulder flexion, shoulder abduction, elbow flexion, wrist flexion). ICCs suggested good concurrent validity for elbow flexion between the Kinect and Dartfish and between the Kinect and Motion Analysis. Good concurrent validity was revealed for maximum height, hand-axle relationship, and maximum area (0.92-0.95) between the Kinect and Dartfish and maximum height and hand-axle relationship (0.89-0.96) between the Kinect and Motion Analysis. Analysis of variance revealed significant differences (p < 0.05) in maximum length between Dartfish (mean 58.76 cm) and the Kinect (40.16 cm). Results pose promising research and clinical implications for propulsion assessment and overuse injury prevention with the application of current findings to future technology.

Across-subject calibration of an instrumented glove to measure hand movement for clinical purposes

Motion capture of all degrees of freedom of the hand collected during performance of daily living activities remains challenging. Instrumented gloves are an attractive option because of their higher ease of use. However, subject-specific calibration of gloves is lengthy and has limitations for individuals with disabilities. Here, a calibration procedure is presented, consisting in the recording of just a simple hand position so as to allow capture of the kinematics of 16 hand joints during daily life activities even in case of severe injured hands. 'across-subject gains' were obtained by averaging the gains obtained from a detailed subject-specific calibration involving 44 registrations that was repeated three times on multiple days to 6 subjects. In additional 4 subjects, joint angles that resulted from applying the 'across-subject calibration' or the subject-specific calibration were compared. Global errors associated with the 'across-subject calibration' relative to the detailed, subject-specific protocol were small (bias: 0.49°; precision: 4.45°) and comparable to those that resulted from repeating the detailed protocol with the same subject on multiple days (0.36°; 3.50°). Furthermore, in one subject, performance of the 'across-subject calibration' was directly compared to another fast calibration method, expressed relative to a videogrammetric protocol as a gold-standard, yielding better results.

Classification of forearm and finger motions using electromyogram and arm-shape-changes

Robot arms for humanoid are widely developed for medical, welfare and education use. Surface electromyogram (sEMG) signals which are the electrical signals obtained from surface of human skin using electrodes have been mainly used for classification of hand motions. However, it is difficult to classify detailed motions such as finger motions and wrist pronation or supination. Moreover, Kinect is an integration sensor device which can capture human joints movement. It also has been widely used for recognition of body motions in many fields. However, it has some problems such as setting of camera and restriction of detection range. In this study, we propose an advanced method of motion classification by combining arm-shape-changes with sEMG to classify the detailed motions. Arm-shape-changes are forearm deformation caused by a bulge of muscle when subjects move an arm or a finger. Experimental results showed classification accuracies of 90% or more in wrist pronation and supination which are difficult to classify using only sEMG signals. As the result, our method could classify the detailed motions and contribute to expansion of classifiable hand motions.

Upper-limb stroke rehabilitation using electrode-array based functional electrical stimulation with sensing and control innovations

Functional electrical stimulation (FES) has shown effectiveness in restoring upper-limb movement post-stroke when applied to assist participants' voluntary intention during repeated, motivating tasks. Recent clinical trials have used advanced controllers that precisely adjust FES to assist functional reach and grasp tasks with FES applied to three muscle groups, showing significant reduction in impairment. The system reported in this paper advances the state-of-the-art by: (1) integrating an FES electrode array on the forearm to assist complex hand and wrist gestures; (2) utilising non-contact depth cameras to accurately record the arm, hand and wrist position in 3D; and (3) employing an interactive touch table to present motivating virtual reality (VR) tasks. The system also uses iterative learning control (ILC), a model-based control strategy which adjusts the applied FES based on the tracking error recorded on previous task attempts. Feasibility of the system has been evaluated in experimental trials with 2 unimpaired participants and clinical trials with 4 hemiparetic, chronic stroke participants. The stroke participants attended 17, 1 hour training sessions in which they performed functional tasks, such as button pressing using the touch table and closing a drawer. Stroke participant results show that the joint angle error norm reduced by an average of 50.3% over 6 attempts at each task when assisted by FES.

Deducing the reachable space from fingertip positions

The reachable space of the hand has received significant interests in the past from relevant medical researchers and health professionals. The reachable space was often computed from the joint angles acquired from a motion capture system such as gloves or markers attached to each bone of the finger. However, the contact between the hand and device can cause difficulties particularly for hand with injuries, burns or experiencing certain dermatological conditions. This paper introduces an approach to find the reachable space of the hand in a non-contact measurement form utilizing the Leap Motion Controller. The approach is based on the analysis of each position in the motion path of the fingertip acquired by the Leap Motion Controller. For each position of the fingertip, the inverse kinematic problem was solved under the physiological multiple constraints of the human hand to find a set of all possible configurations of three finger joints. Subsequently, all the sets are unified to form a set of all possible configurations specific for that motion. Finally, a reachable space is computed from the configuration corresponding to the complete extension and the complete flexion of the finger joint angles in this set.

A Non-Contact Measurement System for the Range of Motion of the Hand

An accurate and standardised tool to measure the active range of motion (ROM) of the hand is essential to any progressive assessment scenario in hand therapy practice. Goniometers are widely used in clinical settings for measuring the ROM of the hand. However, such measurements have limitations with regard to inter-rater and intra-rater reliability and involve direct physical contact with the hand, possibly increasing the risk of transmitting infections. The system proposed in this paper is the first non-contact measurement system utilising Intel Perceptual Technology and a Senz3D Camera for measuring phalangeal joint angles. To enhance the accuracy of the system, we developed a new approach to achieve the total active movement without measuring three joint angles individually. An equation between the actual spacial position and measurement value of the proximal inter-phalangeal joint was established through the measurement values of the total active movement, so that its actual position can be inferred. Verified by computer simulations, experimental results demonstrated a significant improvement in the calculation of the total active movement and successfully recovered the actual position of the proximal inter-phalangeal joint angles. A trial that was conducted to examine the clinical applicability of the system involving 40 healthy subjects confirmed the practicability and consistency in the proposed system. The time efficiency conveyed a stronger argument for this system to replace the current practice of using goniometers.

Evaluation of hand motion capture protocol using static computed tomography images: application to an instrumented glove

There has been a marked increase in the use of hand motion capture protocols in the past 20 yr. However, their absolute accuracies and precisions remain unclear. The purpose of this technical brief was to present a method for evaluating the accuracy and precision of the joint angles determined by a hand motion capture protocol using simultaneously collected static computed tomography (CT) images. The method consists of: (i) recording seven functional postures using both the motion capture protocol and a CT scanner; (ii) obtaining principal axes of the bones in each method; (iii) calculating the flexion angle at each joint for each method as the roll angle of the composite, sequential, roll-pitch-yaw rotations relating the orientation of the distal bone to the proximal bone; and (iv) comparing corresponding joint angle measurements. For demonstration, we applied the method to a Cyberglove protocol. Accuracy and precision of the instrumented-glove protocol were calculated as the mean and standard deviation, respectively, of the differences between the angles determined from the Cyberglove output and the CT images across the seven postures. Implementation in one subject highlighted substantial errors, especially for the distal joints of the fingers. This technical note both clearly demonstrates the need for future work and introduces a solid, technical approach with the potential to improve the current state of such assessments in our field.

Imitation of Dynamic Walking With BSN for Humanoid Robot

Humanoid robots have been used in a wide range of applications including entertainment, healthcare, and assistive living. In these applications, the robots are expected to perform a range of natural body motions, which can be either preprogrammed or learnt from human demonstration. This paper proposes a strategy for imitating dynamic walking gait for a humanoid robot by formulating the problem as an optimization process. The human motion data are recorded with an inertial sensor-based motion tracking system (Biomotion+). Joint angle trajectories are obtained from the transformation of the estimated posture. Key locomotion frames corresponding to gait events are chosen from the trajectories. Due to differences in joint structures of the human and robot, the joint angles at these frames need to be optimized to satisfy the physical constraints of the robot while preserving robot stability. Interpolation among the optimized angles is needed to generate continuous angle trajectories. The method is validated using a NAO humanoid robot, with results demonstrating the effectiveness of the proposed strategy for dynamic walking.

Motor Rehabilitation Using Kinect: A Systematic Review

BACKGROUND

Interactive systems are being developed with the intention to help in the engagement of patients on various therapies. Amid the recent technological advances, Kinect™ from Microsoft (Redmond, WA) has helped pave the way on how user interaction technology facilitates and complements many clinical applications. In order to examine the actual status of Kinect developments for rehabilitation, this article presents a systematic review of articles that involve interactive, evaluative, and technical advances related to motor rehabilitation.

MATERIALS AND METHODS

Systematic research was performed in the IEEE Xplore and PubMed databases using the key word combination "Kinect AND rehabilitation" with the following inclusion criteria: (1) English language, (2) page number >4, (3) Kinect system for assistive interaction or clinical evaluation, or (4) Kinect system for improvement or evaluation of the sensor tracking or movement recognition. Quality assessment was performed by QualSyst standards.

RESULTS

In total, 109 articles were found in the database research, from which 31 were included in the review: 13 were focused on the development of assistive systems for rehabilitation, 3 in evaluation, 3 in the applicability category, 7 on validation of Kinect anatomic and clinical evaluation, and 5 on improvement techniques. Quality analysis of all included articles is also presented with their respective QualSyst checklist scores.

CONCLUSIONS

Research and development possibilities and future works with the Kinect for rehabilitation application are extensive. Methodological improvements when performing studies on this area need to be further investigated.

The measurement of in vivo joint angles during a squat using a single camera markerless motion capture system as compared to a marker based system

Markerless motion capture may have the potential to make motion capture technology widely clinically practical. However, the ability of a single markerless camera system to quantify clinically relevant, lower extremity joint angles has not been studied in vivo. Therefore, the goal of this study was to compare in vivo joint angles calculated using a marker-based motion capture system and a Microsoft Kinect during a squat. Fifteen individuals participated in the study: 8 male, 7 female, height 1.702±0.089m, mass 67.9±10.4kg, age 24±4 years, BMI 23.4±2.2kg/m(2). Marker trajectories and Kinect depth map data of the leg were collected while each subject performed a slow squat motion. Custom code was used to export virtual marker trajectories for the Kinect data. Each set of marker trajectories was utilized to calculate Cardan knee and hip angles. The patterns of motion were similar between systems with average absolute differences of <5 deg. Peak joint angles showed high between-trial reliability with ICC>0.9 for both systems. The peak angles calculated by the marker-based and Kinect systems were largely correlated (r>0.55). These results suggest the data from the Kinect can be post processed in way that it may be a feasible markerless motion capture system that can be used in the clinic.

A Review on Technical and Clinical Impact of Microsoft Kinect on Physical Therapy and Rehabilitation

This paper reviews technical and clinical impact of the Microsoft Kinect in physical therapy and rehabilitation. It covers the studies on patients with neurological disorders including stroke, Parkinson's, cerebral palsy, and MS as well as the elderly patients. Search results in Pubmed and Google scholar reveal increasing interest in using Kinect in medical application. Relevant papers are reviewed and divided into three groups: (1) papers which evaluated Kinect's accuracy and reliability, (2) papers which used Kinect for a rehabilitation system and provided clinical evaluation involving patients, and (3) papers which proposed a Kinect-based system for rehabilitation but fell short of providing clinical validation. At last, to serve as technical comparison to help future rehabilitation design other sensors similar to Kinect are reviewed.

Assessment of movement quality in robot- assisted upper limb rehabilitation after stroke: a review

Studies of stroke patients undergoing robot-assisted rehabilitation have revealed various kinematic parameters describing movement quality of the upper limb. However, due to the different level of stroke impairment and different assessment criteria and interventions, the evaluation of the effectiveness of rehabilitation program is undermined. This paper presents a systematic review of kinematic assessments of movement quality of the upper limb and identifies the suitable parameters describing impairments in stroke patients. A total of 41 different clinical and pilot studies on different phases of stroke recovery utilizing kinematic parameters are evaluated. Kinematic parameters describing movement accuracy are mostly reported for chronic patients with statistically significant outcomes and correlate strongly with clinical assessments. Meanwhile, parameters describing feed-forward sensorimotor control are the most frequently reported in studies on sub-acute patients with significant outcomes albeit without correlation to any clinical assessments. However, lack of measures in coordinated movement and proximal component of upper limb enunciate the difficulties to distinguish the exploitation of joint redundancies exhibited by stroke patients in completing the movement. A further study on overall measures of coordinated movement is recommended.

Complex hand dexterity: a review of biomechanical methods for measuring musical performance

Complex hand dexterity is fundamental to our interactions with the physical, social, and cultural environment. Dexterity can be an expression of creativity and precision in a range of activities, including musical performance. Little is understood about complex hand dexterity or how virtuoso expertise is acquired, due to the versatility of movement combinations available to complete any given task. This has historically limited progress of the field because of difficulties in measuring movements of the hand. Recent developments in methods of motion capture and analysis mean it is now possible to explore the intricate movements of the hand and fingers. These methods allow us insights into the neurophysiological mechanisms underpinning complex hand dexterity and motor learning. They also allow investigation into the key factors that contribute to injury, recovery and functional compensation. The application of such analytical techniques within musical performance provides a multidisciplinary framework for purposeful investigation into the process of learning and skill acquisition in instrumental performance. These highly skilled manual and cognitive tasks present the ultimate achievement in complex hand dexterity. This paper will review methods of assessing instrumental performance in music, focusing specifically on biomechanical measurement and the associated technical challenges faced when measuring highly dexterous activities.