Validity of Robot-Based Assessments of Upper Extremity Function

Artificial Intelligence in Evaluation of Permanent Impairment: New Operational Frontiers

Artificial intelligence (AI) and machine learning (ML) span multiple disciplines, including the medico-legal sciences, also with reference to the concept of disease and disability. In this context, the International Classification of Diseases, Injuries, and Causes of Death (ICD) is a standard for the classification of diseases and related problems developed by the World Health Organization (WHO), and it represents a valid tool for statistical and epidemiological studies. Indeed, the International Classification of Functioning, Disability, and Health (ICF) is outlined as a classification that aims to describe the state of health of people in relation to their existential spheres (social, family, work). This paper lays the foundations for proposing an operating model for the use of AI in the assessment of impairments with the aim of making the information system as homogeneous as possible, starting from the main coding systems of the reference pathologies and functional damages. Providing a scientific basis for the understanding and study of health, as well as establishing a common language for the assessment of disability in its various meanings through AI systems, will allow for the improvement and standardization of communication between the various expert users.

Overview of the role of robots in upper limb disabilities rehabilitation: a scoping review

BACKGROUND

Neuromotor rehabilitation and improvement of upper limb functions are necessary to improve the life quality of patients who have experienced injuries or have pathological outcomes. Modern approaches, such as robotic-assisted rehabilitation can help to improve rehabilitation processes and thus improve upper limb functions. Therefore, the aim of this study was to investigate the role of robots in upper limb disability improvement and rehabilitation.

METHODS

This scoping review was conducted by search in PubMed, Web of Science, Scopus, and IEEE (January 2012- February 2022). Articles related to upper limb rehabilitation robots were selected. The methodological quality of all the included studies will be appraised using the Mixed Methods Appraisal Tool (MMAT). We used an 18-field data extraction form to extract data from articles and extracted the information such as study year, country, type of study, purpose, illness or accident leading to disability, level of disability, assistive technologies, number of participants in the study, sex, age, rehabilitated part of the upper limb using a robot, duration and frequency of treatment, methods of performing rehabilitation exercises, type of evaluation, number of participants in the evaluation process, duration of intervention, study outcomes, and study conclusions. The selection of articles and data extraction was made by three authors based on inclusion and exclusion criteria. Disagreements were resolved through consultation with the fifth author. Inclusion criteria were articles involving upper limb rehabilitation robots, articles about upper limb disability caused by any illness or injury, and articles published in English. Also, articles involving other than upper limb rehabilitation robots, robots related to rehabilitation of diseases other than upper limb, systematic reviews, reviews, and meta-analyses, books, book chapters, letters to the editor, and conference papers were also excluded. Descriptive statistics methods (frequency and percentage) were used to analyses the data.

RESULTS

We finally included 55 relevant articles. Most of the studies were done in Italy (33.82%). Most robots were used to rehabilitate stroke patients (80%). About 60.52% of the studies used games and virtual reality rehabilitate the upper limb disabilities using robots. Among the 14 types of applied evaluation methods, "evaluation and measurement of upper limb function and dexterity" was the most applied evaluation method. "Improvement in musculoskeletal functions", "no adverse effect on patients", and "Safe and reliable treatment" were the most cited outcomes, respectively.

CONCLUSIONS

Our findings show that robots can improve musculoskeletal functions (musculoskeletal strength, sensation, perception, vibration, muscle coordination, less spasticity, flexibility, and range of motion) and empower people by providing a variety of rehabilitation capabilities.

Predicting post-stroke motor recovery of upper extremity using clinical variables and performance assays: A prospective cohort study protocol

BACKGROUND AND PURPOSE

Measurement of movement quality is essential to distinguish motor recovery patterns and optimize rehabilitation strategies post-stroke. Recently, the Stroke Recovery and Rehabilitation Roundtable Taskforce (SRRR) recommended four kinetic and kinematic performance assays to measure upper extremity (UE) movements and distinguish behavioral restitution and compensation mechanisms early post-stroke. The purpose of this study is to develop and validate a prediction model to analyze the added prognostic value of performance assays over clinical variables assessed up to 1-month post stroke for predicting recovery of UE motor impairment, capacity and quality of movement (QoM) measured at 3 months post-stroke onset.

METHODS

In this prospective cohort study, 120 stroke survivors will be recruited within seven days post-stroke. Candidate predictors such as baseline characteristics, demographics and performance assays as per SRRR recommendations along with tonic stretch reflex threshold will be measured up to 1-month post-stroke. Upper extremity motor recovery will be evaluated in terms of motor impairment (Fugl-Meyer assessment for UE), UE capacity measured with Action Research Arm Test (ARAT) and QoM (movement smoothness in the form of peak metrics [PM]) assessed with a reach-to-grasp-to-mouth task (mimicking a drinking task) at 3 months post-stroke. Three multivariable linear regression models will be developed to predict factors responsible for the outcomes of Fugl-Meyer assessment for upper extremity (FM-UE), ARAT and movement quality. The developed models will be internally validated using a split-sample method.

DISCUSSION

This study will provide a validated prediction model inclusive of clinical and performance assays that may assist in prediction of UE motor recovery. Predicting the amount of recovery and differentiating between behavioral restitution and compensation (as reflected by the FM-UE, QoM and ARAT) would enable us in realistic goal formation and planning rehabilitation. It would also help in encouraging patients to partake in early post-stroke rehabilitation thus improving the recovery potential.

Can kinematic parameters of 3D reach-to-target movements be used as a proxy for clinical outcome measures in chronic stroke rehabilitation? An exploratory study

BACKGROUND

Despite numerous trials investigating robot-assisted therapy (RT) effects on upper-extremity (UE) function after stroke, few have explored the relationship between three-dimensional (3D) reach-to-target kinematics and clinical outcomes. The objectives of this study were to 1) investigate the correlation between kinematic parameters of 3D reach-to-target movements and UE clinical outcome measures, and 2) examine the degree to which differences in kinematic parameters across individuals can account for differences in clinical outcomes in response to RT.

METHODS

Ten chronic stroke survivors participated in a pilot RT intervention (eighteen 1-h sessions) integrating cognitive skills training and a home-action program. Clinical outcome measures and kinematic parameters of 3D reach-to-target movements were collected pre- and post-intervention. The correlation between clinical outcomes and kinematic parameters was investigated both cross-sectionally and longitudinally (i.e., changes in response to the intervention). Changes in clinical outcomes and kinematic parameters were tested for significance in both group and subject-by-subject analyses. Potential associations between individual differences in kinematic parameters and differences in clinical outcomes were examined.

RESULTS

Moderate-to-strong correlation was found between clinical measures and specific kinematic parameters when examined cross-sectionally. Weaker correlation coefficients were found longitudinally. Group analyses revealed significant changes in clinical outcome measures in response to the intervention; no significant group changes were observed in kinematic parameters. Subject-by-subject analyses revealed changes with moderate-to-large effect size in the kinematics of 3D reach-to-target movements pre- vs. post-intervention. Changes in clinical outcomes and kinematic parameters varied widely across participants.

CONCLUSIONS

Large variability was observed across subjects in response to the intervention. The correlation between changes in kinematic parameters and clinical outcomes in response to the intervention was variable and not strong across parameters, suggesting no consistent change in UE motor strategies across participants. These results highlight the need to investigate the response to interventions at the individual level. This would enable the identification of clusters of individuals with common patterns of change in response to an intervention, providing an opportunity to use cluster-specific kinematic parameters as a proxy of clinical outcomes.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT02747433 . Registered on April 21st, 2016.

Robotic tests for position sense and movement discrimination in the upper limb reveal that they each are highly reproducible but not correlated in healthy individuals

BACKGROUND

Robotic technologies for neurological assessment provide sensitive, objective measures of behavioural impairments associated with injuries or disease such as stroke. Previous robotic tasks to assess proprioception typically involve single limbs or in some cases both limbs. The challenge with these approaches is that they often rely on intact motor function and/or working memory to remember/reproduce limb position, both of which can be impaired following stroke. Here, we examine the feasibility of a single-arm Movement Discrimination Threshold (MDT) task to assess proprioception by quantifying thresholds for sensing passive limb movement without vision. We use a staircase method to adjust movement magnitude based on subject performance throughout the task in order to reduce assessment time. We compare MDT task performance to our previously-designed Arm Position Matching (APM) task. Critically, we determine test-retest reliability of each task in the same population of healthy controls.

METHOD

Healthy participants (N = 21, age = 18-22 years) completed both tasks in the End-Point Kinarm robot. In the MDT task the robot moved the dominant arm left or right and participants indicated the direction moved. Movement displacement was systematically adjusted (decreased after correct answers, increased after incorrect) until the Discrimination Threshold was found. In the APM task, the robot moved the dominant arm and participants "mirror-matched" with the non-dominant arm.

RESULTS

Discrimination Threshold for direction of arm displacement in the MDT task ranged from 0.1-1.3 cm. Displacement Variability ranged from 0.11-0.71 cm. Test-retest reliability of Discrimination Threshold based on ICC confidence intervals was moderate to excellent (range, ICC = 0.78 [0.52-0.90]). Interestingly, ICC values for Discrimination Threshold increased to 0.90 [0.77-0.96] (good to excellent) when the number of trials was reduced to the first 50. Most APM parameters had ICC's above 0.80, (range, ICC = [0.86-0.88]) with the exception of variability (ICC = 0.30). Importantly, no parameters were significantly correlated across tasks as Spearman rank correlations across parameter-pairings ranged from - 0.27 to 0.30.

CONCLUSIONS

The MDT task is a feasible and reliable task, assessing movement discrimination threshold in ~ 17 min. Lack of correlation between the MDT and a position-matching task (APM) indicates that these tasks assess unique aspects of proprioception that are not strongly related in young, healthy individuals.

Systematic Review on Kinematic Assessments of Upper Limb Movements After Stroke

Background and Purpose- Assessing upper limb movements poststroke is crucial to monitor and understand sensorimotor recovery. Kinematic assessments are expected to enable a sensitive quantification of movement quality and distinguish between restitution and compensation. The nature and practice of these assessments are highly variable and used without knowledge of their clinimetric properties. This presents a challenge when interpreting and comparing results. The purpose of this review was to summarize the state of the art regarding kinematic upper limb assessments poststroke with respect to the assessment task, measurement system, and performance metrics with their clinimetric properties. Subsequently, we aimed to provide evidence-based recommendations for future applications of upper limb kinematics in stroke recovery research. Methods- A systematic search was conducted in PubMed, Embase, CINAHL, and IEEE Xplore. Studies investigating clinimetric properties of applied metrics were assessed for risk of bias using the Consensus-Based Standards for the Selection of Health Measurement Instruments checklist. The quality of evidence for metrics was determined according to the Grading of Recommendations Assessment, Development, and Evaluation approach. Results- A total of 225 studies (N=6197) using 151 different kinematic metrics were identified and allocated to 5 task and 3 measurement system groups. Thirty studies investigated clinimetrics of 62 metrics: reliability (n=8), measurement error (n=5), convergent validity (n=22), and responsiveness (n=2). The metrics task/movement time, number of movement onsets, number of movement ends, path length ratio, peak velocity, number of velocity peaks, trunk displacement, and shoulder flexion/extension received a sufficient evaluation for one clinimetric property. Conclusions- Studies on kinematic assessments of upper limb sensorimotor function are poorly standardized and rarely investigate clinimetrics in an unbiased manner. Based on the available evidence, recommendations on the assessment task, measurement system, and performance metrics were made with the goal to increase standardization. Further high-quality studies evaluating clinimetric properties are needed to validate kinematic assessments, with the long-term goal to elucidate upper limb sensorimotor recovery poststroke. Clinical Trial Registration- URL: https://www.crd.york.ac.uk/prospero/ . Unique identifier: CRD42017064279.

Kinematic Parameters for Tracking Patient Progress during Upper Limb Robot-Assisted Rehabilitation: An Observational Study on Subacute Stroke Subjects

BACKGROUND

Upper limb robot-assisted therapy (RT) provides intensive, repetitive, and task-specific treatment, and its efficacy for stroke survivors is well established in literature. Biomechanical data from robotic devices has been widely employed for patient's assessment, but rarely it has been analysed for tracking patient progress during RT. The goal of this retrospective study is to analyse built-in kinematic data registered by a planar end-effector robot for assessing the time course of motor recovery and patient's workspace exploration skills. A comparison of subjects having mild and severe motor impairment has been also conducted. For that purpose, kinematic data recorded by a planar end-effector robot have been processed for investigating how motor performance in executing point-to-point trajectories with different directions changes during RT.

METHODS

Observational retrospective study of 68 subacute stroke patients who conducted 20 daily sessions of upper limb RT with the InMotion 2.0 (Bionik Laboratories, USA): planar point-to-point reaching tasks with an "assist as needed" strategy. The following kinematic parameters (KPs) were computed for each subject and for each point-to-point trajectory executed during RT: movement accuracy, movement speed, number of peak speed, and task completion time. The Wilcoxon signed-rank tests were used with clinical outcomes. the Friedman test and post hoc Conover's test (Bonferroni's correction) were applied to KPs. A secondary data analysis has been conducted by comparing patients having different severities of motor impairment. The level of significance was set at p value < 0.05.

RESULTS

At the RT onset, the movements were less accurate and smoothed, and showed higher times of execution than those executed at the end of treatment. The analysis of the time course of KPs highlighted that RT seems to improve the motor function mainly in the first sessions of treatment: most KPs show significant intersession differences during the first 5/10 sessions. Afterwards, no further significant variations occurred. The ability to perform movements away from the body and from the hemiparetic side remains more challenging. The results obtained from the data stratification show significant differences between subjects with mild and severe motor impairment.

CONCLUSION

Significant improvements in motor performance were registered during the time course of upper limb RT in subacute stroke patients. The outcomes depend on movement direction and motor impairment and pave the way to optimize healthcare resources and to design patient-tailored rehabilitative protocols.

Validation of computerized square-drawing based evaluation of motor function in patients with stroke

Human-administered clinical scales are commonly used for quantifying motor performance and determining the course of therapy in post-stroke individuals. Computerized methods aim to improve consistency, resolution and duration of patients' evaluation. The objective of this study was to test the validity of computerized square-drawing test (DT) for assessment of shoulder and elbow function by using novel set of DT-based kinematic measures and explore their relation with Wolf Motor Function Test (WMFT) scoring. Forty-seven stroke survivors were tested before and after the rehabilitation program. DT involved drawing a square in horizontal plane using a mechanical manipulandum and a digitizing board. Depending on the initial classification of patients into low or high performance groups, the two different outcome metrics were derived from DT kinematic data for evaluation of each group. Linear regression models applied to map DT outcome values to WMFT scores for both groups resulted with high correlation coefficients and low mean absolute prediction error. In conclusion, we have identified a set of kinematic measures suitable for fast and objective motor function evaluation and functional classification, strongly correlating with WMFT score in post-stroke individuals. The results support validation of square-drawing motor function assessment, encouraging its use in clinical settings.

Nonsurgical Treatment Options for Upper Limb Spasticity

There are many nonsurgical treatment options for patients with upper limb spasticity. This article presents an algorithmic approach to management, encompassing evidence-based rehabilitation therapies, medications, and promising new orthotic and robotic innovations.

Smallest real differences for robotic measures of upper extremity function after stroke: Implications for tracking recovery

Introduction

Measurements from upper limb rehabilitation robots could guide therapy progression, if a robotic assessment’s measurement error was small enough to detect changes occurring on a time scale of a few days. To guide this determination, this study evaluated the smallest real differences of robotic measures, and of clinical outcome assessments predicted from these measures.

Methods

A total of nine older chronic stroke survivors took part in 12-week study with an upper-limb end-effector robot. Fourteen robotic measures were extracted, and used to predict Fugl-Meyer Assessment-Upper Extremity (FMA-UE) and Action Research Arm Test (ARAT) scores using multilinear regression. Smallest real differences and intraclass correlation coefficients were computed for the robotic measures and predicted clinical outcomes, using data from seven baseline sessions.

Results

Smallest real differences of robotic measures ranged from 8.8% to 26.9% of the available range. Smallest real differences of predicted clinical assessments varied widely depending on the regression model (1.3 to 36.2 for FMA-UE, 1.8 to 59.7 for ARAT), and were not strongly related to a model’s predictive performance or to the smallest real differences of the model inputs. Models with acceptable predictive performance as well as low smallest real differences were identified.

Conclusions

Smallest real difference evaluations suggest that using robotic assessments to guide therapy progression is feasible.

Reliability, validity and discriminant ability of the instrumental indices provided by a novel planar robotic device for upper limb rehabilitation

BACKGROUND

In the last few years, there has been an increasing interest in the use of robotic devices to objectively quantify motor performance of patients after brain damage. Although these robot-derived measures can potentially add meaningful information about the patient's dexterity, as well as be used as outcome measurements after the rehabilitation treatment, they need to be validated before being used in clinical practice. The present work aims to evaluate the reliability, the validity and the discriminant ability of the metrics provided by a novel robotic device for upper limb rehabilitation.

METHODS

Forty-eight patients with sub-acute stroke and 40 age-matched healthy subjects were involved in this study. Clinical evaluation included: Fugl-Meyer Assessment for the upper limb, Action Research Arm Test, and Barthel Index. Robotic evaluation of the upper limb performance consisted of 14 measures of motor ability quantifying the dexterity in performing planar reaching movements. Patients were evaluated twice, one day apart, to assess the reliability of the robotic metrics, using the Intraclass Correlation Coefficient. Validity was assessed by analyzing the correlation of the robotic metrics with the clinical scales, by means of the Spearman's Correlation Coefficient. Finally, the ability of the robotic metrics to distinguish between patients with stroke and healthy subjects was investigated with t-tests and the Effect Size.

RESULTS

Reliability was found to be excellent for 12 measures and from moderate to good for the remaining 2. Most of the robotic indices were strongly correlated with the clinical scales, while a few showed a moderate correlation and only one was not correlated with the Barthel Index and weakly correlated with the remain two. Finally, all but one the provided metrics were able to discriminate between the two groups, with large effect sizes for most of them.

CONCLUSION

We found that all the robotic indices except one provided by a novel robotic device for upper limb rehabilitation are reliable, sensitive and strongly correlated both with motor and disability clinical scales. Therefore, this device is suitable as evaluation tool for the upper limb motor performance of patients with sub-acute stroke in clinical practice.

TRIAL REGISTRATION

NCT02879279 .