An adaptive mixed reality training system for stroke rehabilitation

Artificial intelligence in elderly healthcare: A scoping review

The ageing population has led to a surge in the adoption of artificial intelligence (AI) technologies in elderly healthcare worldwide. However, in the advancement of AI technologies, there is currently a lack of clarity about the types and roles of AI technologies in elderly healthcare. This scoping review aimed to provide a comprehensive overview of AI technologies in elderly healthcare by exploring the types of AI technologies employed, and identifying their roles in elderly healthcare based on existing studies. A total of 10 databases were searched for this review, from January 1 2000 to July 31 2022. Based on the inclusion criteria, 105 studies were included. The AI devices utilized in elderly healthcare were summarised as robots, exoskeleton devices, intelligent homes, AI-enabled health smart applications and wearables, voice-activated devices, and virtual reality. Five roles of AI technologies were identified: rehabilitation therapists, emotional supporters, social facilitators, supervisors, and cognitive promoters. Results showed that the impact of AI technologies on elderly healthcare is promising and that AI technologies are capable of satisfying the unmet care needs of older adults and demonstrating great potential in its further development in this area. More well-designed randomised controlled trials are needed in the future to validate the roles of AI technologies in elderly healthcare.

Clinical feasibility and preliminary outcomes of a novel mixed reality system to manage phantom pain: a pilot study

Background

To assess the clinical feasibility of a virtual mirror therapy system in a pilot sample of patients with phantom pain.

Methods

Our Mixed reality system for Managing Phantom Pain (Mr. MAPP) mirrors the preserved limb to visualize the amputated limb virtually and perform exercises. Seven patients with limb loss and phantom pain agreed to participate and received the system for 1-month home use. Outcome measures were collected at baseline and 1 month.

Results

Four (of seven recruited) participants completed the study, which was temporarily suspended due to COVID-19 restrictions. At 1 month, in-game data showed a positive trend, but pain scores showed no clear trends. Functioning scores improved for 1 participant.

Conclusions

Mr. MAPP is feasible and has the potential to improve pain and function in patients with phantom pain.

Trial registration

Clinical Trials Registration, NCT04529083

Supplementary Information

The online version contains supplementary material available at 10.1186/s40814-022-01187-w.

Technology-Based Compensation Assessment and Detection of Upper Extremity Activities of Stroke Survivors: Systematic Review

Background

Upper extremity (UE) impairment affects up to 80% of stroke survivors and accounts for most of the rehabilitation after discharge from the hospital release. Compensation, commonly used by stroke survivors during UE rehabilitation, is applied to adapt to the loss of motor function and may impede the rehabilitation process in the long term and lead to new orthopedic problems. Intensive monitoring of compensatory movements is critical for improving the functional outcomes during rehabilitation.

Objective

This review analyzes how technology-based methods have been applied to assess and detect compensation during stroke UE rehabilitation.

Methods

We conducted a wide database search. All studies were independently screened by 2 reviewers (XW and YF), with a third reviewer (BY) involved in resolving discrepancies. The final included studies were rated according to their level of clinical evidence based on their correlation with clinical scales (with the same tasks or the same evaluation criteria). One reviewer (XW) extracted data on publication, demographic information, compensation types, sensors used for compensation assessment, compensation measurements, and statistical or artificial intelligence methods. Accuracy was checked by another reviewer (YF). Four research questions were presented. For each question, the data were synthesized and tabulated, and a descriptive summary of the findings was provided. The data were synthesized and tabulated based on each research question.

Results

A total of 72 studies were included in this review. In all, 2 types of compensation were identified: disuse of the affected upper limb and awkward use of the affected upper limb to adjust for limited strength, mobility, and motor control. Various models and quantitative measurements have been proposed to characterize compensation. Body-worn technology (25/72, 35% studies) was the most used sensor technology to assess compensation, followed by marker-based motion capture system (24/72, 33% studies) and marker-free vision sensor technology (16/72, 22% studies). Most studies (56/72, 78% studies) used statistical methods for compensation assessment, whereas heterogeneous machine learning algorithms (15/72, 21% studies) were also applied for automatic detection of compensatory movements and postures.

Conclusions

This systematic review provides insights for future research on technology-based compensation assessment and detection in stroke UE rehabilitation. Technology-based compensation assessment and detection have the capacity to augment rehabilitation independent of the constant care of therapists. The drawbacks of each sensor in compensation assessment and detection are discussed, and future research could focus on methods to overcome these disadvantages. It is advised that open data together with multilabel classification algorithms or deep learning algorithms could benefit from automatic real time compensation detection. It is also recommended that technology-based compensation predictions be explored.

Review of Real-Time Biomechanical Feedback Systems in Sport and Rehabilitation

Real-time biomechanical feedback (BMF) is a relatively new area of research. The potential of using advanced technology to improve motion skills in sport and accelerate physical rehabilitation has been demonstrated in a number of studies. This paper provides a literature review of BMF systems in sports and rehabilitation. Our motivation was to examine the history of the field to capture its evolution over time, particularly how technologies are used and implemented in BMF systems, and to identify the most recent studies showing novel solutions and remarkable implementations. We searched for papers in three research databases: Scopus, Web of Science, and PubMed. The initial search yielded 1167 unique papers. After a rigorous and challenging exclusion process, 144 papers were eventually included in this report. We focused on papers describing applications and systems that implement a complete real-time feedback loop, which must include the use of sensors, real-time processing, and concurrent feedback. A number of research questions were raised, and the papers were studied and evaluated accordingly. We identified different types of physical activities, sensors, modalities, actuators, communications, settings and end users. A subset of the included papers, showing the most perspectives, was reviewed in depth to highlight and present their innovative research approaches and techniques. Real-time BMF has great potential in many areas. In recent years, sensors have been the main focus of these studies, but new types of processing devices, methods, and algorithms, actuators, and communication technologies and protocols will be explored in more depth in the future. This paper presents a broad insight into the field of BMF.

Smoothness metrics for reaching performance after stroke. Part 1: which one to choose?

Background

Smoothness is commonly used for measuring movement quality of the upper paretic limb during reaching tasks after stroke. Many different smoothness metrics have been used in stroke research, but a ‘valid’ metric has not been identified. A systematic review and subsequent rigorous analysis of smoothness metrics used in stroke research, in terms of their mathematical definitions and response to simulated perturbations, is needed to conclude whether they are valid for measuring smoothness. Our objective was to provide a recommendation for metrics that reflect smoothness after stroke based on: (1) a systematic review of smoothness metrics for reaching used in stroke research, (2) the mathematical description of the metrics, and (3) the response of metrics to simulated changes associated with smoothness deficits in the reaching profile.

Methods

The systematic review was performed by screening electronic databases using combined keyword groups Stroke, Reaching and Smoothness. Subsequently, each metric identified was assessed with mathematical criteria regarding smoothness: (a) being dimensionless, (b) being reproducible, (c) being based on rate of change of position, and (d) not being a linear transform of other smoothness metrics. The resulting metrics were tested for their response to simulated changes in reaching using models of velocity profiles with varying reaching distances and durations, harmonic disturbances, noise, and sub-movements. Two reaching tasks were simulated; reach-to-point and reach-to-grasp. The metrics that responded as expected in all simulation analyses were considered to be valid.

Results

The systematic review identified 32 different smoothness metrics, 17 of which were excluded based on mathematical criteria, and 13 more as they did not respond as expected in all simulation analyses. Eventually, we found that, for reach-to-point and reach-to-grasp movements, only Spectral Arc Length (SPARC) was found to be a valid metric.

Conclusions

Based on this systematic review and simulation analyses, we recommend the use of SPARC as a valid smoothness metric in both reach-to-point and reach-to-grasp tasks of the upper limb after stroke. However, further research is needed to understand the time course of smoothness measured with SPARC for the upper limb early post stroke, preferably in longitudinal studies.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12984-021-00949-6.

3D mixed-reality visualization of medical imaging data as a supporting tool for innovative, minimally invasive surgery for gastrointestinal tumors and systemic treatment as a new path in personalized treatment of advanced cancer diseases

Background

The purpose of this study was to investigate the potential of a combination of 3D mixed-reality visualization of medical images using CarnaLife Holo (MedApp, Poland) system as a supporting tool for innovative, minimally invasive surgery/irreversible electroporation—IRA, Nano-Knife), microwave ablation (MWA)/for advanced gastrointestinal tumors. Eight liver and pancreatic tumor treatments were performed. In all of the patients undergoing laparoscopy or open surgery volume and margin were estimated by preoperative visualization. In all patients, neoplastic lesions were considered unresectable by standard methods.

Methods

Preoperative CT or MRI were transformed into holograms and displayed thanks to the HoloLens 2. During operation, the surgeon’s field of view was augmented with a 3D model of the patient’s relevant structures.

Results

The intraoperative hologram contributed to better presentation of tumor size and locations, more precise setting of needles used to irreversible electroporation and for determining ablation line in case of liver metastases. Surgeons could easily compare the real patient's anatomy to holographic visualization just before the operations.

Conclusions

The combination of 3D mixed-reality visualization using CarnaLife Holo with IRA, MWA and next systemic treatment (chemotherapy) might be a new way in personalized treatment of advanced cancers.

Upper Limb Home-Based Robotic Rehabilitation During COVID-19 Outbreak

The coronavirus disease (COVID-19) outbreak requires rapid reshaping of rehabilitation services to include patients recovering from severe COVID-19 with post-intensive care syndromes, which results in physical deconditioning and cognitive impairments, patients with comorbid conditions, and other patients requiring physical therapy during the outbreak with no or limited access to hospital and rehabilitation centers. Considering the access barriers to quality rehabilitation settings and services imposed by social distancing and stay-at-home orders, these patients can be benefited from providing access to affordable and good quality care through home-based rehabilitation. The success of such treatment will depend highly on the intensity of the therapy and effort invested by the patient. Monitoring patients' compliance and designing a home-based rehabilitation that can mentally engage them are the critical elements in home-based therapy's success. Hence, we study the state-of-the-art telerehabilitation frameworks and robotic devices, and comment about a hybrid model that can use existing telerehabilitation framework and home-based robotic devices for treatment and simultaneously assess patient's progress remotely. Second, we comment on the patients' social support and engagement, which is critical for the success of telerehabilitation service. As the therapists are not physically present to guide the patients, we also discuss the adaptability requirement of home-based telerehabilitation. Finally, we suggest that the reformed rehabilitation services should consider both home-based solutions for enhancing the activities of daily living and an on-demand ambulatory rehabilitation unit for extensive training where we can monitor both cognitive and motor performance of the patients remotely.

Patterns of enhancement in paretic shoulder kinematics after stroke with musical cueing

Musical cueing has been widely utilised in post-stroke motor rehabilitation; however, the kinematic evidence on the effects of musical cueing is sparse. Further, the element-specific effects of musical cueing on upper-limb movements have rarely been investigated. This study aimed to kinematically quantify the effects of no auditory, rhythmic auditory, and melodic auditory cueing on shoulder abduction, holding, and adduction in patients who had experienced hemiparetic stroke. Kinematic data were obtained using inertial measurement units embedded in wearable bands. During the holding phase, melodic auditory cueing significantly increased the minimum Euler angle and decreased the range of motion compared with the other types of cueing. Further, the root mean square error in the angle measurements was significantly smaller and the duration of movement execution was significantly shorter during the holding phase when melodic auditory cueing was provided than when the other types of cueing were used. These findings indicated the important role of melodic auditory cueing for enhancing movement positioning, variability, and endurance. This study provides the first kinematic evidence on the effects of melodic auditory cueing on kinematic enhancement, thus suggesting the potential use of pitch-related elements in psychomotor rehabilitation.

Design of Virtual Reality-Enabled Surface Electromyogram-Triggered Grip Exercise Platform

Adequate grip ability is important for effective execution of daily living activities. Neurological disorders like stroke that result in muscle weakness, limited strength and poor control often lead to reduced grip ability in the affected limb. Conventional rehabilitation for grip training is often monotonous and subjective. Technology-assisted Virtual Reality (VR)-based rehabilitation offers a motivating environment to the participants for rehabilitation. However, the existing systems need specialized set-up architecture, thereby limiting their accessibility. Furthermore, these systems quantify the functional grip ability based on task performance, and do not explore physiological basis of grip ability. In this work, we develop a VR-based rehabilitation platform integrated with physiology-sensitive biofeedback. The developed platform, Gripx makes use of features extracted from sEMG data collected from upper limb muscles to adaptively provide audio-visual biofeedback through a VR environment. We compare task based performance, physiological indices and clinical measures to evaluate the potential of Gripx. The results of our study with 8 healthy and 12 post-stroke participants show the potential of Gripx to contribute to grip rehabilitation over multiple exposures. This approach of integrating VR-based task design with physiology-sensitive biofeedback helps patients to better assess their physiological responses and enhance the efficacy of rehabilitation.

sEMG-Based Trunk Compensation Detection in Rehabilitation Training

Stroke patients often use trunk to compensate for impaired upper limb motor function during upper limb rehabilitation training, which results in a reduced rehabilitation training effect. Detecting trunk compensations can improve the effect of rehabilitation training. This study investigates the feasibility of a surface electromyography-based trunk compensation detection (sEMG-bTCD) method. Five healthy subjects and nine stroke subjects with cognitive and comprehension skills were recruited to participate in the experiments. The sEMG signals from nine superficial trunk muscles were collected during three rehabilitation training tasks (reach-forward-back, reach-side-to-side, and reach-up-to-down motions) without compensation and with three common trunk compensations [lean-forward (LF), trunk rotation (TR), and shoulder elevation (SE)]. Preprocessing like filtering, active segment detection was performed and five time domain features (root mean square, variance, mean absolute value (MAV), waveform length, and the fourth order autoregressive model coefficient) were extracted from the collected sEMG signals. Excellent TCD performance was achieved in healthy participants by using support vector machine (SVM) classifier (LF: accuracy = 94.0%, AUC = 0.97, F1 = 0.94; TR: accuracy = 95.8%, AUC = 0.99, F1 = 0.96; SE: accuracy = 100.0%, AUC = 1.00, F1 = 1.00). By using SVM classifier, TCD performance in stroke participants was also obtained (LF: accuracy = 74.8%, AUC = 0.90, F1 = 0.73; TR: accuracy = 67.1%, AUC = 0.85, F1 = 0.71; SE: accuracy = 91.3%, AUC = 0.98, F1 = 0.90). Compared with the methods based on cameras or inertial sensors, better detection performance was obtained in both healthy and stroke participants. The results demonstrated the feasibility of the sEMG-bTCD method, and it helps to prompt the stroke patients to correct their incorrect posture, thereby improving the effectiveness of rehabilitation training.

Meta-analyzability Concerns in Meta-analyses

Many studies to date have conducted a meta-analysis on a mix of effectiveness and superiority studies. This methodological flaw will lead to difficulties in interpreting the results. We addressed this issue in this article, illustrated our point with a simulated experiment, and re-analyzed a recent meta-analysis study based on the effectiveness-superiority dichotomy to provide a real-world correlate of our point of view.

Investigating the feasibility and acceptability of real-time visual feedback in reducing compensatory motions during self-administered stroke rehabilitation exercises: A pilot study with chronic stroke survivors

INTRODUCTION

Homework-based rehabilitation programs can help stroke survivors restore upper extremity function. However, compensatory motions can develop without therapist supervision, leading to sub-optimal recovery. We developed a visual feedback system using a live video feed or an avatar reflecting users' movements so users are aware of compensations. This pilot study aimed to evaluate validity (how well the avatar characterizes different types of compensations) and acceptability of the system.

METHODS

Ten participants with chronic stroke performed upper-extremity exercises under three feedback conditions: none, video, and avatar. Validity was evaluated by comparing agreement on compensations annotated using video and avatar images. A usability survey was administered to participants after the experiment to obtain information on acceptability.

RESULTS

There was substantial agreement between video and avatar images for shoulder elevation and hip extension (Cohen's κ: 0.6-0.8) and almost perfect agreement for trunk rotation and flexion (κ: 0.80-1). Acceptability was low due to lack of corrective prompts and occasional noise with the avatar display. Most participants suggested that an automatic compensation detection feature with visual and auditory cuing would improve the system.

CONCLUSION

The avatar characterized four types of compensations well. Future work will involve increasing sensitivity for shoulder elevation and implementing a method to detect compensations.

Three-Dimensional Augmented Reality System for Balance and Mobility Rehabilitation in the Elderly: A Randomized Controlled Trial

We attempted to evaluate the clinical efficiency of a novel three-dimensional interactive augmented reality system (3D-ARS) for balance and mobility rehabilitation. This system enables participant training with a realistic 3D interactive balance exercise and assessing movement parameters and joint angles by using a kinetic sensor system. We performed a randomized controlled trial in a general hospital. Thirty-six participants (age, 56-76 years) who could independently walk and stand on one leg were recruited. The participants were randomly assigned to either group. The control group (n = 18) underwent a conventional physical fitness program such as lower-extremity strengthening and balance training thrice per week for 1 month. The experimental group (n = 18) experienced 3D-ARS training thrice per week (1 session = 30 minutes) for 4 weeks. Training comprised a balloon game for hip exercise, cave game for knee exercise, and rhythm game for one-leg balance exercise. Lower-extremity clinical scale scores, fall index, and automatic balance score were measured by using Tetrax^® posturography before, during, and after training. Significant group (3D-ARS vs. control) × time (before and after exercise) interaction effect was observed for Berg balance scale (BBS) scores (p = 0.04) and timed-up-and-go (TUG; p < 0.001). Overall improvements occurred in stability index, weight distribution index, fall risk index, and Fourier transformations index of posturography for both groups. However, score changes were significantly greater in the 3D-ARS group. Significant group × time interaction effect was observed for the fall risk index. This demonstrates that the 3D-ARS system can improve balance in the elderly more effectively.

Kinect-based virtual rehabilitation and evaluation system for upper limb disorders: A case study

OBJECTIVE

To help patients with disabilities of the arm and shoulder recover the accuracy and stability of movements, a novel and simple virtual rehabilitation and evaluation system called the Kine-VRES system was developed using Microsoft Kinect.

METHODS

First, several movements and virtual tasks were designed to increase the coordination, control and speed of the arm movements. The movements of the patients were then captured using the Kinect sensor, and kinematics-based interaction and real-time feedback were integrated into the system to enhance the motivation and self-confidence of the patient. Finally, a quantitative evaluation method of upper limb movements was provided using the recorded kinematics during hand-to-hand movement.

RESULTS

A preliminary study of this rehabilitation system indicates that the shoulder movements of two participants with ataxia became smoother after three weeks of training (one hour per day).

CONCLUSION

This case study demonstrated the effectiveness of the designed system, which could be promising for the rehabilitation of patients with upper limb disorders.

Realization of real-time X-ray stereoscopic vision during interventional procedures

During interventional procedures, the deficiencies of nonstereoscopic vision increase the difficulty of identifying the anteroposterior direction and pathways of vessels. Therefore, achieving real-time stereoscopic vision during interventional procedures is meaningful. Pairs of X-ray images were captured with identical parameter settings, except for different rotation angles (represented as the α angle). The resulting images at these α angles were used as left-eye and right-eye views and were horizontally merged into single left-right 3D images. Virtual reality (VR) glasses were used for achieving stereo vision. Pairs of X-ray images from four angiographies with different α angles (1.8-3.4°) were merged into left-right 3D images. Observation with VR glasses can produce realistic stereo views of vascular anatomical structure. The results showed that the optimal α angles accepted by the brain for generating stereo vision were within a narrow range (approximately 1.4-4.1°). Subsequent tests showed that during transcatheter arterial chemoembolization, 3D X-ray stereoscopic images provided significantly improved spatial discrimination and convenience for identifying the supply vessels of a liver tumor and its anteroposterior direction compared with plain X-ray images (all P < 0.01). Real-time X-ray stereoscopic vision can be easily achieved via the straightforward method described herein and has the potential to benefit patients during interventional procedures.

Movement Sonification in Stroke Rehabilitation

Stroke often affects arm functions and thus impairs patients' daily activities. Recently, several studies have shown that additional movement acoustics can enhance motor perception and motor control. Therefore, a new method has been developed that allows providing auditory feedback about arm movement trajectories in real-time for motor rehabilitation after stroke. The present article describes the study protocol for a randomized, controlled, examiner, and patient blinded superiority trial (German Clinical Trials Register, www.drks.de, DRKS00011419), in which the method will be applied to 13 subacute stroke patients with hemiparesis during 12 sessions of 30 min each as additional feedback during the regular movement therapy. As primary outcome, a significant pre-post-change in the Box and Block Test is expected that exceeds the performance increase of 13 patients who will be provided with sham-acoustics. Possible limitations of the method as well as the study design are discussed.

Usability evaluation of low-cost virtual reality hand and arm rehabilitation games

The emergence of lower-cost motion tracking devices enables home-based virtual reality rehabilitation activities and increased accessibility to patients. Currently, little documentation on patients' expectations for virtual reality rehabilitation is available. This study surveyed 10 people with stroke for their expectations of virtual reality rehabilitation games. This study also evaluated the usability of three lower-cost virtual reality rehabilitation games using a survey and House of Quality analysis. The games (kitchen, archery, and puzzle) were developed in the laboratory to encourage coordinated finger and arm movements. Lower-cost motion tracking devices, the P5 Glove and Microsoft Kinect, were used to record the movements. People with stroke were found to desire motivating and easy-to-use games with clinical insights and encouragement from therapists. The House of Quality analysis revealed that the games should be improved by obtaining evidence for clinical effectiveness, including clinical feedback regarding improving functional abilities, adapting the games to the user's changing functional ability, and improving usability of the motion-tracking devices. This study reports the expectations of people with stroke for rehabilitation games and usability analysis that can help guide development of future games.

What do randomized controlled trials say about virtual rehabilitation in stroke? A systematic literature review and meta-analysis of upper-limb and cognitive outcomes

BACKGROUND

Virtual-reality based rehabilitation (VR) shows potential as an engaging and effective way to improve upper-limb function and cognitive abilities following a stroke. However, an updated synthesis of the literature is needed to capture growth in recent research and address gaps in our understanding of factors that may optimize training parameters and treatment effects.

METHODS

Published randomized controlled trials comparing VR to conventional therapy were retrieved from seven electronic databases. Treatment effects (Hedge's g) were estimated using a random effects model, with motor and functional outcomes between different protocols compared at the Body Structure/Function, Activity, and Participation levels of the International Classification of Functioning.

RESULTS

Thirty-three studies were identified, including 971 participants (492 VR participants). VR produced small to medium overall effects (g = 0.46; 95% CI: 0.33-0.59, p < 0.01), above and beyond conventional therapies. Small to medium effects were observed on Body Structure/Function (g = 0.41; 95% CI: 0.28-0.55; p < 0.01) and Activity outcomes (g = 0.47; 95% CI: 0.34-0.60, p < 0.01), while Participation outcomes failed to reach significance (g = 0.38; 95% CI: -0.29-1.04, p = 0.27). Superior benefits for Body Structure/Function (g = 0.56) and Activity outcomes (g = 0.62) were observed when examining outcomes only from purpose-designed VR systems. Preliminary results (k = 4) suggested small to medium effects for cognitive outcomes (g = 0.41; 95% CI: 0.28-0.55; p < 0.01). Moderator analysis found no advantage for higher doses of VR, massed practice training schedules, or greater time since injury.

CONCLUSION

VR can effect significant gains on Body Structure/Function and Activity level outcomes, including improvements in cognitive function, for individuals who have sustained a stroke. The evidence supports the use of VR as an adjunct for stroke rehabilitation, with effectiveness evident for a variety of platforms, training parameters, and stages of recovery.

Identifying compensatory movement patterns in the upper extremity using a wearable sensor system

OBJECTIVE

Movement impairments such as those due to stroke often result in the nervous system adopting atypical movements to compensate for movement deficits. Monitoring these compensatory patterns is critical for improving functional outcomes during rehabilitation. The purpose of this study was to test the feasibility and validity of a wearable sensor system for detecting compensatory trunk kinematics during activities of daily living.

APPROACH

Participants with no history of neurological impairments performed reaching and manipulation tasks with their upper extremity, and their movements were recorded by a wearable sensor system and validated using a motion capture system. Compensatory movements of the trunk were induced using a brace that limited range of motion at the elbow.

MAIN RESULTS

Our results showed that the elbow brace elicited compensatory movements of the trunk during reaching tasks but not manipulation tasks, and that a wearable sensor system with two sensors could reliably classify compensatory movements (~90% accuracy).

SIGNIFICANCE

These results show the potential of the wearable system to assess and monitor compensatory movements outside of a lab setting.

The use and effect of video game design theory in the creation of game-based systems for upper limb stroke rehabilitation

Upper limb exercise is often neglected during post-stroke rehabilitation. Video games have been shown to be useful in providing environments in which patients can practise repetitive, functionally meaningful movements, and in inducing neuroplasticity. The design of video games is often focused upon a number of fundamental principles, such as reward, goals, challenge and the concept of meaningful play, and these same principles are important in the design of games for rehabilitation. Further to this, there have been several attempts for the strengthening of the relationship between commercial game design and rehabilitative game design, the former providing insight into factors that can increase motivation and engagement with the latter. In this article, we present an overview of various game design principles and the theoretical grounding behind their presence, in addition to attempts made to utilise these principles in the creation of upper limb stroke rehabilitation systems and the outcomes of their use. We also present research aiming to move the collaborative efforts of designers and therapists towards a model for the structured design of these games and the various steps taken concerning the theoretical classification and mapping of game design concepts with intended cognitive and motor outcomes.

Short-Duration and Intensive Training Improves Long-Term Reaching Performance in Individuals With Chronic Stroke

Previous studies have shown that multiple sessions of reach training lead to long-term improvements in movement time and smoothness in individuals post-stroke. Yet such long-term training regimens are often difficult to implement in actual clinical settings. In this study, we evaluated the long-term and generalization effects of short-duration and intensive reach training in 16 individuals with chronic stroke and mild to moderate impairments. Participants performed 2 sessions of unassisted intensive reach training, with 600 movements per session, and with display of performance-based feedback after each movement. The participants' trunks were restrained with a belt to avoid compensatory movements. Training resulted in significant and durable (1 month) improvements in movement time (20.4% on average) and movement smoothness (22.7% on average). The largest improvements occurred in individuals with the largest initial motor impairments. In addition, training induced generalization to nontrained targets, which persisted in 1-day and in 1-month retention tests. Finally, there was a significant improvement in the Box and Block test from baseline to 1-month retention test (23% on average). Thus, short-duration and intensive reach training can lead to generalized and durable benefits in individuals with chronic stroke and mild to moderate impairments.

Adaptive personalized training games for individual and collaborative rehabilitation of people with multiple sclerosis

Any rehabilitation involves people who are unique individuals with their own characteristics and rehabilitation needs, including patients suffering from Multiple Sclerosis (MS). The prominent variation of MS symptoms and the disease severity elevate a need to accommodate the patient diversity and support adaptive personalized training to meet every patient's rehabilitation needs. In this paper, we focus on integrating adaptivity and personalization in rehabilitation training for MS patients. We introduced the automatic adjustment of difficulty levels as an adaptation that can be provided in individual and collaborative rehabilitation training exercises for MS patients. Two user studies have been carried out with nine MS patients to investigate the outcome of this adaptation. The findings showed that adaptive personalized training trajectories have been successfully provided to MS patients according to their individual training progress, which was appreciated by the patients and the therapist. They considered the automatic adjustment of difficulty levels to provide more variety in the training and to minimize the therapists involvement in setting up the training. With regard to social interaction in the collaborative training exercise, we have observed some social behaviors between the patients and their training partner which indicated the development of social interaction during the training.

A computerized recognition system for the home-based physiotherapy exercises using an RGBD camera

Computerized recognition of the home based physiotherapy exercises has many benefits and it has attracted considerable interest among the computer vision community. However, most methods in the literature view this task as a special case of motion recognition. In contrast, we propose to employ the three main components of a physiotherapy exercise (the motion patterns, the stance knowledge, and the exercise object) as different recognition tasks and embed them separately into the recognition system. The low level information about each component is gathered using machine learning methods. Then, we use a generative Bayesian network to recognize the exercise types by combining the information from these sources at an abstract level, which takes the advantage of domain knowledge for a more robust system. Finally, a novel postprocessing step is employed to estimate the exercise repetitions counts. The performance evaluation of the system is conducted with a new dataset which contains RGB (red, green, and blue) and depth videos of home-based exercise sessions for commonly applied shoulder and knee exercises. The proposed system works without any body-part segmentation, bodypart tracking, joint detection, and temporal segmentation methods. In the end, favorable exercise recognition rates and encouraging results on the estimation of repetition counts are obtained.

A feasibility study of an upper limb rehabilitation system using Kinect and computer games

A new low-cost system for rehabilitation of the impaired upper limb for stroke survivors is presented. A computer game was developed specifically for this purpose and the user's impaired upper extremity is tracked using a downward-pointed Kinect, an inexpensive motion capture system commercially available from Microsoft. A Kalman filter was implemented to reduce data jittering. Patients are required to move their impaired arm, sliding it on top of a transparent support, in order to play the game. The game is personalized to the patient through specific settings that adapt to the patient's range of motion and motor control at the start of the game as well as performance during the game. The final score is proportional to the arm's movement speed. A feasibility study was carried out with one stroke survivor. The game was played for ten days and usability surveys were answered before and after the study. The patient was engaged with the game, found it easy to understand and reported willingness to use it in the home environment and enjoyment of the use in the clinic.

A computational framework for constructing interactive feedback for assisting motor learning

New motion capture technologies are allowing detailed, precise and complete monitoring of movement through real-time kinematic analysis. However, a clinically relevant understanding of movement impairment through kinematic analysis requires the development of computational models that integrate clinical expertise in the weighing of the kinematic parameters. The resulting kinematics based measures of movement impairment would further need to be integrated with existing clinical measures of activity disability. This is a challenging process requiring computational solutions that can extract correlations within and between three diverse data sets: human driven assessment of body function, kinematic based assessment of movement impairment and human driven assessment of activity. We propose to identify and characterize different sensorimotor control strategies used by normal individuals and by hemiparetic stroke survivors acquiring a skilled motor task. We will use novel quantitative approaches to further our understanding of how human motor function is coupled to multiple and simultaneous modes of feedback. The experiments rely on a novel interactive tasks environment developed by our team in which subjects are provided with rich auditory and visual feedback of movement variables to drive motor learning. Our proposed research will result in a computational framework for applying virtual information to assist motor learning for complex tasks that require coupling of proprioception, vision audio and haptic cues. We shall use the framework to devise a computational tool to assist with therapy of stroke survivors. This tool will utilize extracted relationships in a pre-clinical setting to generate effective and customized rehabilitation strategies.

A low cost, adaptive mixed reality system for home-based stroke rehabilitation

This paper presents a novel, low-cost, real-time adaptive multimedia environment for home-based upper extremity rehabilitation of stroke survivors. The primary goal of this system is to provide an interactive tool with which the stroke survivor can sustain gains achieved within the clinical phase of therapy and increase the opportunity for functional recovery. This home-based mediated system has low cost sensing, off the shelf components for the auditory and visual feedback, and remote monitoring capability. The system is designed to continue active learning by reducing dependency on real-time feedback and focusing on summary feedback after a single task and sequences of tasks. To increase system effectiveness through customization, we use data from the training strategy developed by the therapist at the clinic for each stroke survivor to drive automated system adaptation at the home. The adaptation includes changing training focus, selecting proper feedback coupling both in real-time and in summary, and constructing appropriate dialogues with the stroke survivor to promote more efficient use of the system. This system also allows the therapist to review participant's progress and adjust the training strategy weekly.

Adaptive Hierarchical Control for the Muscle Strength Training of Stroke Survivors in Robot-Aided Upper-Limb Rehabilitation

Muscle strength training for stroke patients is of vital importance for helping survivors to progressively restore muscle strength and improve the performance of their activities in daily living (ADL). An adaptive hierarchical therapy control framework which integrates the patient's real biomechanical state estimation with task-performance quantitative evaluation is proposed. Firstly, a high-level progressive resistive supervisory controller is designed to determine the resistive force base for each training session based on the patient's online task-performance evaluation. Then, a low-level adaptive resistive force triggered controller is presented to further regulate the interactive resistive force corresponding to the patient's real-time biomechanical state – characterized by the patient's bio-damping and bio-stiffness in the course of one training session, so that the patient is challenged in a moderate but engaging and motivating way. Finally, a therapeutic robot system using a Barrett WAM™ compliant manipulator is set up. We recruited eighteen inpatient and outpatient stroke participants who were randomly allocated in experimental (robot-aided) and control (conventional physical therapy) groups and enrolled for sixteen weeks of progressive resistance training. The preliminary results show that the proposed therapy control strategies can enhance the recovery of strength and motor control ability.

Exploring the bases for a mixed reality stroke rehabilitation system, Part II: design of interactive feedback for upper limb rehabilitation

Background

Few existing interactive rehabilitation systems can effectively communicate multiple aspects of movement performance simultaneously, in a manner that appropriately adapts across various training scenarios. In order to address the need for such systems within stroke rehabilitation training, a unified approach for designing interactive systems for upper limb rehabilitation of stroke survivors has been developed and applied for the implementation of an Adaptive Mixed Reality Rehabilitation (AMRR) System.

Results

The AMRR system provides computational evaluation and multimedia feedback for the upper limb rehabilitation of stroke survivors. A participant's movements are tracked by motion capture technology and evaluated by computational means. The resulting data are used to generate interactive media-based feedback that communicates to the participant detailed, intuitive evaluations of his performance. This article describes how the AMRR system's interactive feedback is designed to address specific movement challenges faced by stroke survivors. Multimedia examples are provided to illustrate each feedback component. Supportive data are provided for three participants of varying impairment levels to demonstrate the system's ability to train both targeted and integrated aspects of movement.

Conclusions

The AMRR system supports training of multiple movement aspects together or in isolation, within adaptable sequences, through cohesive feedback that is based on formalized compositional design principles. From preliminary analysis of the data, we infer that the system's ability to train multiple foci together or in isolation in adaptable sequences, utilizing appropriately designed feedback, can lead to functional improvement. The evaluation and feedback frameworks established within the AMRR system will be applied to the development of a novel home-based system to provide an engaging yet low-cost extension of training for longer periods of time.

Mixed reality rehabilitation for stroke survivors promotes generalized motor improvements

This paper presents results from a clinical study of stroke survivors using an adaptive, mixed-reality rehabilitation (AMRR) system for reach and grasp therapy. The AMRR therapy provides audio and visual feedback on the therapy task, based on detailed motion capture, that places the movement in an abstract, artistic context. This type of environment promotes the generalizability of movement strategies, which is shown through kinematic improvements on an untrained reaching task and higher clinical scale scores, in addition to kinematic improvements in the trained task.

Utilizing media arts principles for developing effective interactive neurorehabilitation systems

This paper discusses how interactive neurorehabilitation systems can increase their effectiveness through systematic integration of media arts principles and practice. Media arts expertise can foster the development of complex yet intuitive extrinsic feedback displays that match the inherent complexity and intuitive nature of motor learning. Abstract, arts-based feedback displays can be powerful metaphors that provide re-contextualization, engagement and appropriate reward mechanisms for mature adults. Such virtual feedback displays must be seamlessly integrated with physical components to produce mixed reality training environments that promote active, generalizable learning. The proposed approaches are illustrated through examples from mixed reality rehabilitation systems developed by our team.