Interdisciplinary concepts for design and implementation of mixed reality interactive neurorehabilitation systems for stroke

Capturing Upper Body Kinematics and Localization with Low-Cost Sensors for Rehabilitation Applications

For upper extremity rehabilitation, quantitative measurements of a person’s capabilities during activities of daily living could provide useful information for therapists, including in telemedicine scenarios. Specifically, measurements of a person’s upper body kinematics could give information about which arm motions or movement features are in need of additional therapy, and their location within the home could give context to these motions. To that end, we present a new algorithm for identifying a person’s location in a region of interest based on a Bluetooth received signal strength (RSS) and present an experimental evaluation of this and a different Bluetooth RSS-based localization algorithm via fingerprinting. We further present algorithms for and experimental results of inferring the complete upper body kinematics based on three standalone inertial measurement unit (IMU) sensors mounted on the wrists and pelvis. Our experimental results for localization find the target location with a mean square error of 1.78 m. Our kinematics reconstruction algorithms gave lower errors with the pelvis sensor mounted on the person’s back and with individual calibrations for each test. With three standalone IMUs, the mean angular error for all of the upper body segment orientations was close to 21 degrees, and the estimated elbow and shoulder angles had mean errors of less than 4 degrees.

Automated Movement Assessment in Stroke Rehabilitation

We are developing a system for long term Semi-Automated Rehabilitation At the Home (SARAH) that relies on low-cost and unobtrusive video-based sensing. We present a cyber-human methodology used by the SARAH system for automated assessment of upper extremity stroke rehabilitation at the home. We propose a hierarchical model for automatically segmenting stroke survivor's movements and generating training task performance assessment scores during rehabilitation. The hierarchical model fuses expert therapist knowledge-based approaches with data-driven techniques. The expert knowledge is more observable in the higher layers of the hierarchy (task and segment) and therefore more accessible to algorithms incorporating high level constraints relating to activity structure (i.e., type and order of segments per task). We utilize an HMM and a Decision Tree model to connect these high level priors to data driven analysis. The lower layers (RGB images and raw kinematics) need to be addressed primarily through data driven techniques. We use a transformer based architecture operating on low-level action features (tracking of individual body joints and objects) and a Multi-Stage Temporal Convolutional Network(MS-TCN) operating on raw RGB images. We develop a sequence combining these complimentary algorithms effectively, thus encoding the information from different layers of the movement hierarchy. Through this combination, we produce a robust segmentation and task assessment results on noisy, variable and limited data, which is characteristic of low cost video capture of rehabilitation at the home. Our proposed approach achieves 85% accuracy in per-frame labeling, 99% accuracy in segment classification and 93% accuracy in task completion assessment. Although the methodology proposed in this paper applies to upper extremity rehabilitation using the SARAH system, it can potentially be used, with minor alterations, to assist automation in many other movement rehabilitation contexts (i.e., lower extremity training for neurological accidents).

Challenges in developing a magnetic resonance-compatible haptic hand-controller for neurosurgical training

A haptic device is an actuated human-machine interface utilized by an operator to dynamically interact with a remote environment. This interaction could be virtual (virtual reality) or physical such as using a robotic arm. To date, different mechanisms have been considered to actuate the haptic device to reflect force feedback from the remote environment. In a low-force environment or limited working envelope, the control of some actuation mechanisms such as hydraulic and pneumatic may be problematic. In the development of a haptic device, challenges include limited space, high accuracy or resolution, limitations in kinematic and dynamic solutions, points of singularity, dexterity as well as control system development/design. Furthermore, the haptic interface designed to operate in a magnetic resonance imaging environment adds additional challenges related to electromagnetic interference, static/variable magnetic fields, and the use of magnetic resonance-compatible materials. Such a device would allow functional magnetic resonance imaging to obtain information on the subject's brain activity while performing a task. When used for surgical trainees, functional magnetic resonance imaging could provide an assessment of surgical skills. In this application, the trainee, located supine within the magnet bore while observing the task environment on a graphical user interface, uses a low-force magnetic resonance-compatible haptic device to perform virtual surgical tasks in a limited space. In the quest to develop such a device, this review reports the multiple challenges faced and their potential solutions. The review also investigates efforts toward prototyping such devices and classifies the main components of a magnetic resonance-compatible device including actuation and sensory systems and materials used.

Improvement in Stroke-induced Motor Dysfunction by Music-supported Therapy: A Systematic Review and Meta-analysis

To conduct a meta-analysis of clinical trials that examined the effect of music-supported therapy on stroke-induced motor dysfunction, comprehensive literature searches of PubMed, Embase and the Cochrane Library from their inception to April 2016 were performed. A total of 10 studies (13 analyses, 358 subjects) were included; all had acceptable quality according to PEDro scale score. The baseline differences between the two groups were confirmed to be comparable. Compared with the control group, the standardized mean difference of 9-Hole Peg Test was 0.28 (-0.01, 0.57), 0.64 (0.31, 0.97) in Box and Block Test, 0.47 (0.08, 0.87) in Arm Paresis Score and 0.35 (-0.04, 0.75) in Action Research Arm Test for upper-limb motor function, 0.11 (-0.24, 0.46) in Berg Balance Scale score, 0.09 (-0.36, 0.54) in Fugl-Meyer Assessment score, 0.30 (-0.15, 0.74) in Wolf Motor Function Test, 0.30 (-0.15, 0.74) in Wolf Motor Function time, 0.65 (0.14, 1.16) in Stride length and 0.62 (0.01, 1.24) in Gait Velocity for total motor function, and 1.75 (0.94, 2.56) in Frontal Assessment Battery score for executive function. There was evidence of a positive effect of music-supported therapy, supporting its use for the treatment of stroke-induced motor dysfunction. This study was registered at PRESPERO (CRD42016037106).