Concurrent validity of an immersive virtual reality version of the Box and Block Test to assess manual dexterity among patients with stroke

Technological Features of Immersive Virtual Reality Systems for Upper Limb Stroke Rehabilitation: A Systematic Review

Stroke is the second most common cause of death worldwide, and it greatly impacts the quality of life for survivors by causing impairments in their upper limbs. Due to the difficulties in accessing rehabilitation services, immersive virtual reality (IVR) is an interesting approach to improve the availability of rehabilitation services. This systematic review evaluates the technological characteristics of IVR systems used in the rehabilitation of upper limb stroke patients. Twenty-five publications were included. Various technical aspects such as game engines, programming languages, headsets, platforms, game genres, and technical evaluation were extracted from these papers. Unity 3D and C# are the primary tools for creating IVR apps, while the Oculus Quest (Meta Platforms Technologies, Menlo Park, CA, USA) is the most often used headset. The majority of systems are created specifically for rehabilitation purposes rather than being readily available for purchase (i.e., commercial games). The analysis also highlights key areas for future research, such as game assessment, the combination of hardware and software, and the potential integration incorporation of biofeedback sensors. The study highlights the significance of technological progress in improving the effectiveness and user-friendliness of IVR. It calls for additional research to fully exploit IVR's potential in enhancing stroke rehabilitation results.

Clinical Utility and Usability of the Digital Box and Block Test: Mixed Methods Study

Background

The Box and Block Test (BBT) is a clinical tool used to measure hand dexterity, which is often used for tracking disease progression or the effectiveness of therapy, particularly benefiting older adults and those with neurological conditions. Digitizing the measurement of hand function may enhance the quality of data collection. We have developed and validated a prototype that digitizes this test, known as the digital BBT (dBBT), which automatically measures time and determines and displays the test result.

Objective

This study aimed to investigate the clinical utility and usability of the newly developed dBBT and to collect suggestions for future improvements.

Methods

A total of 4 occupational therapists participated in our study. To evaluate the clinical utility, we compared the dBBT to the BBT across dimensions such as acceptance, portability, energy and effort, time, and costs. We observed therapists using the dBBT as a dexterity measurement tool and conducted a quantitative usability questionnaire using the System Usability Scale (SUS), along with a focus group. Evaluative, structured, and qualitative content analysis was used for the qualitative data, whereas quantitative analysis was applied to questionnaire data. The qualitative and quantitative data were merged and analyzed using a convergent mixed methods approach.

Results

Overall, the results of the evaluative content analysis suggested that the dBBT had a better clinical utility than the original BBT, with ratings of all collected participant statements for the dBBT being 45% (45/99) equal to, 48% (48/99) better than, and 6% (6/99) lesser than the BBT. Particularly in the subcategories "acceptance," "time required for evaluation," and "purchase costs," the dBBT was rated as being better than the original BBT. The dBBT achieved a mean SUS score of 83 (95% CI 76-96). Additionally, several suggested changes to the system were identified.

Conclusions

The study demonstrated an overall positive evaluation of the clinical utility and usability of the dBBT. Valuable insights were gathered for future system iterations. These pioneering results highlight the potential of digitizing hand dexterity assessments.

Effects of Maitland mobilization technique on upper extremity function in stroke survivors with spasticity: An experimental study

BACKGROUND

The recovery of upper limb function is of great significance for stroke patients to regain their self-care ability, yet it is still a difficult point in clinical practice of neurological rehabilitation. This study aimed to investigate the effect of Maitland joint mobilization technique on the recovery of upper extremity function in patients with spasticity after stroke.

METHODS

From August to December 2023, 71 patients with upper extremity flexor spasm after stroke were recruited and randomly divided into experimental group (n = 35) and control group (n = 36). The control group was given conventional rehabilitation treatment, while the experimental group was treated with Maitland mobilization technique treatment of upper extremity joints on the basis of the control group. The experiment lasted for 8 weeks. Participants of the 2 groups were observed for Fugl-Meyer motor assessment-upper extremity (FMA-UE), box and block test (BBT) and Brunnstrom stage, modified Ashworth scale (MAS), and functional independence measure (FIM) at pre- and post-8 weeks study.

RESULTS

There was no significant difference in gender distribution, hemiplegic side, diagnosis, past history, age, duration, body mass index, and mini-mental state examination between the 2 groups (P > .05). After 8 weeks of intervention, both groups showed significant improvement in FMA-UE, Brunnstrom stage, BBT, FIM, and MAS of the shoulder (P < .05); however, there was no significant change in MAS of the elbow, wrist, and finger joints (P > .05). The posttreatment values showed a significant improvement in FMA-UE, BBT, and FIM in the experimental group compared to the control group. Comparing the changes in pretreatment and posttreatment, FMA-UE, BBT, and FIM in the experimental group were significantly improved compared with those in the control group (P < .05).

CONCLUSION

Maitland joint mobilization can improve the motor function of upper extremity and the spasticity of shoulder joint complex in patients with stroke.

Ηand dexterities assessment in stroke patients based on augmented reality and machine learning through a box and block test

A popular and widely suggested measure for assessing unilateral hand motor skills in stroke patients is the box and block test (BBT). Our study aimed to create an augmented reality enhanced version of the BBT (AR-BBT) and evaluate its correlation to the original BBT for stroke patients. Following G-power analysis, clinical examination, and inclusion-exclusion criteria, 31 stroke patients were included in this study. AR-BBT was developed using the Open Source Computer Vision Library (OpenCV). The MediaPipe's hand tracking library uses a palm and a hand landmark machine learning model to detect and track hands. A computer and a depth camera were employed in the clinical evaluation of AR-BBT following the principles of traditional BBT. A strong correlation was achieved between the number of blocks moved in the BBT and the AR-BBT on the hemiplegic side (Pearson correlation = 0.918) and a positive statistically significant correlation (p = 0.000008). The conventional BBT is currently the preferred assessment method. However, our approach offers an advantage, as it suggests that an AR-BBT solution could remotely monitor the assessment of a home-based rehabilitation program and provide additional hand kinematic information for hand dexterities in AR environment conditions. Furthermore, it employs minimal hardware equipment.

Extended reality to assess post-stroke manual dexterity: contrasts between the classic box and block test, immersive virtual reality with controllers, with hand-tracking, and mixed-reality tests

BACKGROUND

Recent technological advancements present promising opportunities to enhance the frequency and objectivity of functional assessments, aligning with recent stroke rehabilitation guidelines. Within this framework, we designed and adapted different manual dexterity tests in extended reality (XR), using immersive virtual reality (VR) with controllers (BBT-VR-C), immersive VR with hand-tracking (BBT-VR-HT), and mixed-reality (MD-MR).

OBJECTIVE

This study primarily aimed to assess and compare the validity of the BBT-VR-C, BBT-VR-HT and MD-MR to assess post-stroke manual dexterity. Secondary objectives were to evaluate reliability, usability and to define arm kinematics measures.

METHODS

A sample of 21 healthy control participants (HCP) and 21 stroke individuals with hemiparesis (IHP) completed three trials of the traditional BBT, the BBT-VR-C, BBT-VR-HT and MD-MR. Content validity of the different tests were evaluated by asking five healthcare professionals to rate the difficulty of performing each test in comparison to the traditional BBT. Convergent validity was evaluated through correlations between the scores of the traditional BBT and the XR tests. Test-retest reliability was assessed through correlations between the second and third trial and usability was assessed using the System Usability Scale (SUS). Lastly, upper limb movement smoothness (SPARC) was compared between IHP and HCP for both BBT-VR test versions.

RESULTS

For content validity, healthcare professionals rated the BBT-VR-HT (0[0-1]) and BBT-MR (0[0-1]) as equally difficult to the traditional BBT, whereas they rated BBT-VR-C as more difficult than the traditional BBT (1[0-2]). For IHP convergent validity, the Pearson tests demonstrated larger correlations between the scores of BBT and BBT-VR-HT (r = 0.94;p < 0.001), and BBT and MD-MR (r = 0.95;p < 0.001) than BBT and BBT-VR-C (r = 0.65;p = 0.001). BBT-VR-HT and MD-MR usability were both rated as excellent, with median SUS scores of 83[57.5-91.3] and 83[53.8-92.5] respectively. Excellent reliability was found for the BBT-VR-C (ICC = 0.96;p < 0.001), BBT-VR-HT (ICC = 0.96;p < 0.001) and BBT-MR (ICC = 0.99;p < 0.001). The usability of the BBT-VR-C was rated as good with a median SUS of 70[43.8-83.8]. Upper limb movements of HCP were significantly smoother than for IHP when completing either the BBT-VR-C (t = 2.05;p = 0.043) and the BBT-VR-HT (t = 5.21;p < 0.001).

CONCLUSION

The different XR manual tests are valid, short-term reliable and usable tools to assess post-stroke manual dexterity.

TRIAL REGISTRATION

https://clinicaltrials.gov/ct2/show/NCT04694833 ; Unique identifier: NCT04694833, Date of registration: 11/24/2020.

Performance-based outcome measures of upper extremity in virtual reality and telerehabilitation: a systematic review

OBJECTIVES

This systematic review aimed to identify and synthesize evidence regarding measurement properties of upper extremity performance-based outcome measures performed in virtual reality or in telerehabilitation to inform clinical applications and research endeavors.

METHODS

Five bibliographic databases, PubMed, Embase, CINAHL, APA PsycINFO, and Scopus, were searched on July 12, 2023. Studies assessing the measurement properties (reliability, validity, responsiveness) of upper extremity performance-based outcome measures in virtual reality and telerehabilitation were eligible for inclusion. The COSMIN risk of bias checklist was used for methodological quality assessment. Study selection, data extraction, and quality assessment were completed by two independent reviewers.

RESULTS

A total of 240 records were identified from the five databases. Nine cross-sectional studies published from 2016 to 2023 were included. Participants included 210 patients with neurological conditions and 184 healthy subjects. In virtual reality studies, four implemented the Box and Block Test, one the Arm Research Action Test, and one the Peg Insertion Test. In telerehabilitation studies, three implemented the Fugl-Meyer Assessment Upper Extremity. For quality assessment, one study was rated inadequate, one was rated doubtful, and all others demonstrated adequate to good quality. Most studies demonstrated good test-retest reliability and concurrent validity to the original in-person assessments.

CONCLUSION

Implementing upper extremity performance-based outcome measures in virtual reality and telerehabilitation is feasible and promising. Further studies are warranted to develop and refine remote assessment paradigms and validate them on a larger scale to inform clinical application and promote digital health in rehabilitation.

Immersive Virtual Reality to Assess Arm Kinematics among Older Adults with and without Major Neurocognitive Disorder - An Exploratory Cross-Sectional Study

Despite the recommendation of improving assessment objectivity and frequency, the use of immersive virtual reality to measure and quantify movement quality remains underexplored. In this study, we aimed to evaluate the reliability, validity and usability of an immersive virtual reality application, KinematicsVR, to assess upper limb kinematics among older adults with and without major neurocognitive disorder. The KinematicsVR involves the drawing of three-dimensional straight lines, circles and squares using a controller in a virtual environment. Twenty-eight older adults with or without major neurocognitive disorder were recruited. Reliability was evaluated through correlations on test-retest and validity through correlations between KinematicsVR variables and other functional tests (TEMPA, BBT-VR and Finger-Nose Test). The usability of the KinematicsVR was assessed with the System Usability Scale questionnaire. Kinematic indexes were compared between eight adults with major neurocognitive disorder and eight matched controls. Results indicated that most variables provided by the KinematicsVR had excellent reliability for tasks involving the drawing of straight lines and circles, but moderate reliability for tasks involving the drawing of squares. Secondary analyses showed that the usability of the application was excellent but few significant and strong correlations were observed between variables of the KinematicsVR and the scores of the TEMPA scale, Finger-Nose Test and BBT-VR. Adults with major neurocognitive disorder, when compared to other older adults, made larger and less linear hand movements. These findings provide perspectives for the use of immersive virtual reality to improve assessment frequency and objectivity through the autonomous measure of upper limb kinematics in older adults.

Comparison of virtual reality to physical box and blocks on cortical an neuromuscualar activations in young adults

The purpose of this study was to assess the changes in neural activations when performing the box and block test (BBT) in virtual reality (VR) compared to the physical BBT. Young healthy participants performed three trials of the BBT with their left and right hands in both the VR BBT, using VR hand controllers, and physical BBT conditions. Electromyography sensors were placed on the upper extremity of both arms and functional near-infrared spectroscopy was used to measure motor cortex activations throughout each condition. While a reduction in BBT score and increased wrist extensor neuromuscular activity is exhibited during the VR condition, there is no statistical difference in motor cortex activation between the two BBT conditions. This work provides a basis for exploring cortical and neuromuscular responses to VR in patient populations.

A Digital Box and Block Test for Hand Dexterity Measurement: Instrument Validation Study

BACKGROUND

The Box and Block Test (BBT) measures unilateral gross manual dexterity and is widely used in clinical settings with a wide range of populations, including older people and clients with neurological disorders.

OBJECTIVE

In this study, we present a newly developed digitized version of the BBT, called the digital BBT (dBBT). The physical design is similar to the original BBT, but the dBBT contains digital electronics that automate the test procedure, timing, and score measurement. The aim of this study is to investigate the validity and reliability of the dBBT.

METHODS

We performed measurements at 2 time points for 29 healthy participants. BBT and dBBT were used at the first measurement time point, and dBBT was used again at the second measurement time point. Concurrent validity was assessed using the correlation between BBT and dBBT, the paired t test, and the Bland-Altman analysis. Test-retest reliability and interrater reliability were examined using the interclass correlation coefficient (ICC) by repeated measures with the dBBT within an interval of 10 days.

RESULTS

Our results showed moderate concurrent validity (r=0.48, P=.008), moderate test-retest reliability (ICC 0.72, P<.001), a standard error of measurement of 3.1 blocks, and the smallest detectable change at a 95% CI of 8.5 blocks. Interrater reliability was moderate with an ICC of 0.67 (P=.02). The Bland-Altman analysis showed sufficient accuracy of the dBBT in comparison with the conventional BBT.

CONCLUSIONS

The dBBT can contribute to objectifying the measurement of gross hand dexterity without losing its important characteristics and is simple to implement.

Visual feedback and age affect upper limb reaching accuracy and kinematics in immersive virtual reality among healthy adults

This cross-sectional study aimed to evaluate the effect of visual feedback, age and movement repetition on the upper limb (UL) accuracy and kinematics during a reaching task in immersive virtual reality (VR). Fifty-one healthy participants were asked to perform 25 trials of a reaching task in immersive VR with and without visual feedback of their hand. They were instructed to place, as accurately and as fast as possible, a controller held in their non-dominant hand in the centre of a virtual red cube of 3 cm side length. For each trial, the end-point error (distance between the tip of the controller and the centre of the cube), a coefficient of linearity (CL), the movement time (MT), and the spectral arc length of the velocity signal (SPARC), which is a movement smoothness index, were calculated. Multivariate analyses of variance were conducted to assess the influence of visual feedback, age and trial repetition on the average end-point error, SPARC, CL and MT, and their time course throughout the 25 trials. Providing visual feedback of the hand reduced average end-point error ( P  < 0.001) and MT ( P  = 0.044), improved SPARC ( P  < 0.001) but did not affect CL ( P  = 0.07). Younger participants obtained a lower mean end-point error ( P  = 0.037), a higher SPARC ( P  = 0.021) and CL ( P  = 0.013). MT was not affected by age ( P  = 0.671). Trial repetition increased SPARC ( P  < 0.001) and CL ( P  < 0.001), and reduced MT ( P  = 0.001) but did not affect end-point error ( P  = 0.608). In conclusion, the results of this study demonstrated that providing visual feedback of the hand and being younger improves UL accuracy and movement smoothness in immersive VR. UL kinematics but not accuracy can be improved with more trial repetitions. These findings could guide the future development of protocols in clinical rehabilitation and research.

Using immersive virtual reality to remotely examine performance differences between dominant and non-dominant hands

Circle drawing may be a useful task to study upper-limb function in patient populations. However, previous studies rely on expensive and bulky robotics to measure performance. For clinics or hospitals with limited budgets and space, this may be unfeasible. Virtual reality (VR) provides a portable and low-cost tool with integrated motion capture. It offers potentially a more feasible medium by which to assess upper-limb motor function. Prior to use with patient populations, it is important to validate and test the capabilities of VR with healthy users. This study examined whether a VR-based circle drawing task, completed remotely using participant's own devices, could capture differences between movement kinematics of the dominant and non-dominant hands in healthy individuals. Participants (n = 47) traced the outline of a circle presented on their VR head-mounted displays with each hand, while the positions of the hand-held controllers were continuously recorded. Although there were no differences observed in the size or roundness of circles drawn with each hand, consistent with prior literature our results did show that the circles drawn with the dominant hand were completed faster than those with the non-dominant hand. This provides preliminary evidence that a VR-based circle drawing task may be a feasible method for detecting subtle differences in function in clinical populations.

Supplementary Information

The online version contains supplementary material available at 10.1007/s10055-023-00794-z.

Laboratory-Based Examination of the Reliability and Validity of Kinematic Measures of Wrist and Finger Function Collected by a Telerehabilitation System in Persons with Chronic Stroke

We have developed the New Jersey Institute of Technology-Home Virtual Rehabilitation System (NJIT-HoVRS) to facilitate intensive, hand-focused rehabilitation in the home. We developed testing simulations with the goal of providing richer information for clinicians performing remote assessments. This paper presents the results of reliability testing examining differences between in-person and remote testing as well as discriminatory and convergent validity testing of a battery of six kinematic measures collected with NJIT-HoVRS. Two different groups of persons with upper extremity impairments due to chronic stroke participated in two separate experiments. Data Collection: All data collection sessions included six kinematic tests collected with the Leap Motion Controller. Measurements collected include hand opening range, wrist extension range, pronation-supination range, hand opening accuracy, wrist extension accuracy, and pronation-supination accuracy. The system usability was evaluated by therapists performing the reliability study using the System Usability Scale. When comparing the in-laboratory collection and the first remote collection, the intra-class correlation coefficients (ICC) for three of the six measurements were above 0.900 and the other three were between 0.500 and 0.900. Two of the first remote collection/second remote collection ICCs were above 0.900, and the other four were between 0.600 and 0.900. The 95% confidence intervals for these ICC were broad, suggesting that these preliminary analyses need to be confirmed by studies with larger samples. The therapist's SUS scores ranged from 70 to 90. The mean was 83.1 (SD = 6.4), which is consistent with industry adoption. There were statistically significant differences in the kinematic scores when comparing unimpaired and impaired UE for all six measures. Five of six impaired hand kinematic scores and five of six impaired/unimpaired hand difference scores demonstrated correlations between 0.400 and 0.700 with UEFMA scores. Reliability for all measures was acceptable for clinical practice. Discriminant and convergent validity testing suggest that scores on these tests may be meaningful and valid. Further testing in a remote setting is necessary to validate this process.

Performing a shortened version of the Action Research Arm Test in immersive virtual reality to assess post-stroke upper limb activity

BACKGROUND

To plan treatment and measure post-stroke recovery, frequent and time-bounded functional assessments are recommended. With increasing needs for neurorehabilitation advances, new technology based methods, such as virtual reality (VR) have emerged. Here, we developed an immersive VR version of the Action Research Arm Test (ARAT-VR) to complement neurorehabilitation.

OBJECTIVE

This study aimed to assess the validity, usability and test-retest reliability of the ARAT-VR among individuals with stroke, healthcare professionals and healthy control subjects (HCS).

METHODS

Among the 19 items of the ARAT, 13 items were selected and developed in immersive VR. 11 healthcare professionals, 30 individuals with stroke, and 25 HCS were recruited. Content validity was assessed by asking healthcare professionals to rate the difficulty of performing each item of the ARAT-VR in comparison to the classical Action Research Arm Test (ARAT-19). Concurrent validity was first measured using correlation (Spearman tests) between the ARAT-VR and ARAT-19 scores for the individuals with stroke, and second through correlation and comparison between the scores of the ARAT-VR and the reduced version of the ARAT (ARAT-13) for both individuals with stroke and HCS (Wilcoxon signed rank tests and Bland-Altman plots). Usability was measured using the System Usability Scale. A part of individuals with stroke and HCS were re-tested following a convenient delay to measure test-retest reliability (Intra-class correlation and Wilcoxon tests).

RESULTS

Regarding the content validity, median difficulty of the 13 ARAT-VR items (0[0 to - 1] to 0[0-1]) evaluated by healthcare professionals was rated as equivalent to the classical ARAT for all tasks except those involving the marbles. For these, the difficulty was rated as superior to the real tasks (1[0-1] when pinching with the thumb-index and thumb-middle fingers, and 1[0-2] when pinching with thumb-ring finger). Regarding the concurrent validity, for paretic hand scores, there were strong correlations between the ARAT-VR and ARAT-13 (r = 0.84), and between the ARAT-VR and ARAT-19 (r = 0.83). Usability (SUS = 82.5[75-90]) and test-retest reliability (ICC = 0.99; p < 0.001) were excellent.

CONCLUSION

The ARAT-VR is a valid, usable and reliable tool that can be used to assess upper limb activity among individuals with stroke, providing potential to increase assessment frequency, remote evaluation, and improve neurorehabilitation. Trial registration https://clinicaltrials.gov/ct2/show/NCT04694833 ; Unique identifier: NCT04694833, Date of registration: 11/24/2020.

New technologies promoting active upper limb rehabilitation after stroke: an overview and network meta-analysis

INTRODUCTION

The primary aim of this work was to summarize and compare the effects of active rehabilitation assisted by new technologies (virtual reality [VR], robot-assisted therapy [RAT] and telerehabilitation [TR)) on upper limb motor function and everyday living activity during the subacute and chronic phases of stroke. The secondary aims were to compare the effects of these technologies according to the intervention design (in addition to or in substitution of conventional therapy), the duration of active rehabilitation and the severity of patients' motor impairments.

EVIDENCE ACQUISITION

Several databases, namely PubMed, Scopus, Embase and Cochrane Library, were searched. Studies were included if they were meta-analyses with a moderate to high level of confidence (assessed with AMSTAR-2) that compared the effects of a new technology promoting active rehabilitation to that of a conventional therapy program among patients with stroke. Network meta-analyses were conducted to compare the effects of the new technologies.

EVIDENCE SYNTHESIS

Eighteen different meta-analyses were selected and fifteen included in the quantitative analysis. In total these 15 meta-analyses were based on 189 different randomized controlled trials. VR (SMD≥0.25; P<0.05), RAT (SMD≥0.29; P≤0.29) and TR (SMD≥-0.08; P≤0.64) were found to be at least as effective as conventional therapy. During the subacute phase, RAT's greatest effect was observed for patients with severe-moderate impairments whereas VR and TR's greatest effects were observed for patients with mild impairments. During the chronic phase, the highest effects were observed for patients with mild impairments, for all studies technologies. Network meta-analyses showed that VR and RAT were both significantly superior to TR in improving motor function during the chronic phase but revealed no significant difference between VR, RAT and TR effectiveness on both motor function (during the subacute phase) and activity (during both chronic and subacute phase).

CONCLUSIONS

This overview provides low-to-moderate evidence that rehabilitation assisted with technologies are at least as effective as conventional therapy for patients with stroke. While VR and RAT seem to be more efficient during the subacute phase, all technologies seem to be as efficient as one another in the chronic phase.