Improvement of Apraxia With Augmented Reality: Influencing Pantomime of Tool Use via Holographic Cues

Ηand dexterities assessment in stroke patients based on augmented reality and machine learning through a box and block test

A popular and widely suggested measure for assessing unilateral hand motor skills in stroke patients is the box and block test (BBT). Our study aimed to create an augmented reality enhanced version of the BBT (AR-BBT) and evaluate its correlation to the original BBT for stroke patients. Following G-power analysis, clinical examination, and inclusion-exclusion criteria, 31 stroke patients were included in this study. AR-BBT was developed using the Open Source Computer Vision Library (OpenCV). The MediaPipe's hand tracking library uses a palm and a hand landmark machine learning model to detect and track hands. A computer and a depth camera were employed in the clinical evaluation of AR-BBT following the principles of traditional BBT. A strong correlation was achieved between the number of blocks moved in the BBT and the AR-BBT on the hemiplegic side (Pearson correlation = 0.918) and a positive statistically significant correlation (p = 0.000008). The conventional BBT is currently the preferred assessment method. However, our approach offers an advantage, as it suggests that an AR-BBT solution could remotely monitor the assessment of a home-based rehabilitation program and provide additional hand kinematic information for hand dexterities in AR environment conditions. Furthermore, it employs minimal hardware equipment.

Robotic arm training in neurorehabilitation enhanced by augmented reality - a usability and feasibility study

BACKGROUND

Robotic therapy and serious gaming support motor learning in neurorehabilitation. Traditional monitor-based gaming outputs cannot adequately represent the third dimension, whereas virtual reality headsets lack the connection to the real world. The use of Augmented Reality (AR) techniques could potentially overcome these issues. The objective of this study was thus to evaluate the usability, feasibility and functionality of a novel arm rehabilitation device for neurorehabilitation (RobExReha system) based on a robotic arm (LBR iiwa, KUKA AG) and serious gaming using the AR headset HoloLens (Microsoft Inc.).

METHODS

The RobExReha system was tested with eleven adult inpatients (mean age: 64.4 ± 11.2 years; diagnoses: 8 stroke, 2 spinal cord injury, 1 Guillain-Barré-Syndrome) who had paretic impairments in their upper limb. Five therapists administered and evaluated the system. Data was compared with a Reference Group (eleven inpatients; mean age: 64.3 ± 9.1 years; diagnoses: 10 stroke, 1 spinal cord injury) who trained with commercially available robotic therapy devices (ArmeoPower or ArmeoSpring, Hocoma AG). Patients used standardized questionnaires for evaluating usability and comfort (Quebec User Evaluation of Satisfaction with assistive technology [QUEST]), workload (Raw Task Load Index [RTLX]) and a questionnaire for rating visual perception of the gaming scenario. Therapists used the QUEST, the System Usability Scale and the short version of the User Experience Questionnaire.

RESULTS

Therapy with the RobExReha system was safe and feasible for patients and therapists, with no serious adverse events being reported. Patients and therapists were generally satisfied with usability. The patients' usability ratings were significantly higher in the Reference Group for two items of the QUEST: reliability and ease of use. Workload (RTLX) ratings did not differ significantly between the groups. Nearly all patients using the RobExReha system perceived the gaming scenario in AR as functioning adequately despite eight patients having impairments in stereoscopic vision. The therapists valued the system's approach as interesting and inventive.

CONCLUSIONS

We demonstrated the clinical feasibility of combining a novel robotic upper limb robot with an AR-serious game in a neurorehabilitation setting. To ensure high usability in future applications, a reliable and easy-to-use system that can be used for task-oriented training should be implemented.

TRIAL REGISTRATION

Ethical approval was obtained and the trial was registered at the German Clinical Trials Register (DRKS00022136).

Limb apraxia in individuals with multiple sclerosis: Is there a role of semi-immersive virtual reality in treating the Cinderella of neuropsychology?

BACKGROUND

Limb apraxia is an acquired cognitive-motor disorder characterized by spatial and temporal disorganization of limb movements, negatively affecting the quality of life of patients, including those with multiple sclerosis (MS). Although recent studies have shown the potential role of VR in increasing cognitive and motor functions, only a few studies have been carried out on the rehabilitation of upper limb apraxia. Hence, our study aims to evaluate the potential efficacy of VR training to improve upper limb ideomotor apraxia in patients with MS.

METHODS

One hundred and six patients, affected by secondary progressive MS, who attended our Robotic and Behavioral Neurorehabilitation Service from March 2019 to February 2020, were enrolled in this study and randomly divided into two groups: the control group (CG: 53 patients) performed traditional therapy whereas the experimental group (EG:53 patients) received training using semi-immersive VR. All patients underwent the same amount of cognitive training, 3 times a week for 8 weeks. They were submitted to a specific neuropsychological assessment before (T0) and after the rehabilitation treatment (T1).

RESULTS

The VR training led to a significant improvement in global cognitive functions, with regard to constructive and ideomotor apraxia. On the contrary, the CG achieved significant improvements only in ideomotor apraxia. Moreover, only in the EG, we observed an improvement in the mood at the end of training.

CONCLUSION

The present study demonstrates that VR rehabilitation can be an effective tool for the treatment of apraxia, which is a neuropsychological problem often underestimated in MS patients. Further studies with long-term follow-up periods are needed to confirm the effect of this promising approach.

The impact of visuospatial perception on distance judgment and depth perception in an Augmented Reality environment in patients after stroke: an exploratory study

BACKGROUND

Augmented Reality (AR)-based interventions are applied in neurorehabilitation with increasing frequency. Depth perception is required for the intended interaction within AR environments. Until now, however, it is unclear whether patients after stroke with impaired visuospatial perception (VSP) are able to perceive depth in the AR environment.

METHODS

Different aspects of VSP (stereovision and spatial localization/visuoconstruction) were assessed in 20 patients after stroke (mean age: 64 ± 14 years) and 20 healthy subjects (HS, mean age: 28 ± 8 years) using clinical tests. The group of HS was recruited to assess the validity of the developed AR tasks in testing stereovision. To measure perception of holographic objects, three distance judgment tasks and one three-dimensionality task were designed. The effect of impaired stereovision on performance in each AR task was analyzed. AR task performance was modeled by aspects of VSP using separate regression analyses for HS and for patients.

RESULTS

In HS, stereovision had a significant effect on the performance in all AR distance judgment tasks (p = 0.021, p = 0.002, p = 0.046) and in the three-dimensionality task (p = 0.003). Individual quality of stereovision significantly predicted the accuracy in each distance judgment task and was highly related to the ability to perceive holograms as three-dimensional (p = 0.001). In stroke-survivors, impaired stereovision had a specific deterioration effect on only one distance judgment task (p = 0.042), whereas the three-dimensionality task was unaffected (p = 0.317). Regression analyses confirmed a lacking impact of patients' quality of stereovision on AR task performance, while spatial localization/visuoconstruction significantly prognosticated the accuracy in distance estimation of geometric objects in two AR tasks.

CONCLUSION

Impairments in VSP reduce the ability to estimate distance and to perceive three-dimensionality in an AR environment. While stereovision is key for task performance in HS, spatial localization/visuoconstruction is predominant in patients. Since impairments in VSP are present after stroke, these findings might be crucial when AR is applied for neurorehabilitative treatment. In order to maximize the therapy outcome, the design of AR games should be adapted to patients' impaired VSP.  Trial registration: The trial was not registered, as it was an observational study.