Design and validation of a novel online platform to support the usability evaluation of wearable robotic devices

Designing for usability: development and evaluation of a portable minimally-actuated haptic hand and forearm trainer for unsupervised stroke rehabilitation

In stroke rehabilitation, simple robotic devices hold the potential to increase the training dosage in group therapies and to enable continued therapy at home after hospital discharge. However, we identified a lack of portable and cost-effective devices that not only focus on improving motor functions but also address sensory deficits. Thus, we designed a minimally-actuated hand training device that incorporates active grasping movements and passive pronosupination, complemented by a rehabilitative game with meaningful haptic feedback. Following a human-centered design approach, we conducted a usability study with 13 healthy participants, including three therapists. In a simulated unsupervised environment, the naive participants had to set up and use the device based on written instructions. Our mixed-methods approach included quantitative data from performance metrics, standardized questionnaires, and eye tracking, alongside qualitative feedback from semi-structured interviews. The study results highlighted the device's overall ease of setup and use, as well as its realistic haptic feedback. The eye-tracking analysis further suggested that participants felt safe during usage. Moreover, the study provided crucial insights for future improvements such as a more intuitive and comfortable wrist fixation, more natural pronosupination movements, and easier-to-follow instructions. Our research underscores the importance of continuous testing in the development process and offers significant contributions to the design of user-friendly, unsupervised neurorehabilitation technologies to improve sensorimotor stroke rehabilitation.

Towards a validated glossary of usability attributes for the evaluation of wearable robotic devices

BACKGROUND

Despite technical advances in the field of wearable robotic devices (WRD), there is still limited user acceptance of these technologies. While usability often comes as a key factor influencing acceptance, there is a scattered landscape of definitions and scopes for the term. To advance usability evaluation, and to integrate usability features as design requirements during technology development, there is a need for benchmarks and shared terminology. These should be easily accessible and implementable by developers.

METHODS

An initial set of usability attributes (UA) was extracted from a literature survey on usability evaluation in WRD. The initial set of attributes was enriched and locally validated with seven developers of WRD through an online survey and a focus group. The locally validated glossary was then externally validated through a globally distributed online survey.

RESULTS

The result is the Robotics Usability Glossary (RUG), a comprehensive glossary of 41 UA validated by 70 WRD developers from 17 countries, ensuring its generalizability. 31 of the UA had high agreement scores among respondents and 27 were considered highly relevant in the field, but only 11 of them had been included as design criteria by the respondents.

CONCLUSIONS

Multiple UA ought to be considered for a comprehensive usability assessment. Usability remains inadequately incorporated into device development, indicating a need for increased awareness and end-user perspective. The RUG can be readily accessed through an online platform, the Interactive Usability Toolbox (IUT), developed to provide context-specific outcome measures and usability evaluation methods. Overall, this effort is an important step towards improving and promoting usability evaluation practices within WRD. It has the potential to pave the way for establishing usability evaluation benchmarks that further endorse the acceptance of WRD.

The most used questionnaires for evaluating the usability of robots and smart wearables: A scoping review

Background

As the field of robotics and smart wearables continues to advance rapidly, the evaluation of their usability becomes paramount. Researchers may encounter difficulty in finding a suitable questionnaire for evaluating the usability of robotics and smart wearables. Therefore, the aim of this study is to identify the most commonly utilized questionnaires for assessing the usability of robots and smart wearables.

Methods

A comprehensive search of databases, including PubMed, Web of Science, and Scopus, was conducted for this scoping review. Two authors performed the selection of articles and data extraction using a 10-field data extraction form. In cases of disagreements, a third author was consulted to reach a consensus. The inclusions were English-language original research articles that utilized validated questionnaires to assess the usability of healthcare robots and smart wearables. The exclusions comprised review articles, non-English publications, studies not focused on usability, those assessing clinical outcomes, articles lacking questionnaire details, and those using non-validated or researcher-made questionnaires. Descriptive statistics methods (frequency and percentage), were employed to analyze the data.

Results

A total of 314 articles were obtained, and after eliminating irrelevant and duplicate articles, a final selection of 50 articles was included in this review. A total of 17 questionnaires were identified to evaluate the usability of robots and smart wearables, with 10 questionnaires specifically for wearables and 7 questionnaires for robots. The System Usability Scale (50%) and Post-Study System Usability Questionnaire (19.44%) were the predominant questionnaires utilized to assess the usability of smart wearables. Moreover, the most commonly used questionnaires for evaluating the usability of robots were the System Usability Scale (56.66%), User Experience Questionnaire (16.66%), and Quebec User Evaluation of Satisfaction with Assistive Technology (10%).

Conclusion

Commonly employed questionnaires serve as valuable tools in assessing the usability of robots and smart wearables, aiding in the refinement and optimization of these technologies for enhanced user experiences. By incorporating user feedback and insights, designers can strive towards creating more intuitive and effective robotic and wearable solutions.

Mixed methods usability evaluation of an assistive wearable robotic hand orthosis for people with spinal cord injury

BACKGROUND

Robotic hand orthoses (RHO) aim to provide grasp assistance for people with sensorimotor hand impairment during daily tasks. Many of such devices have been shown to bring a functional benefit to the user. However, assessing functional benefit is not sufficient to evaluate the usability of such technologies for daily life application. A comprehensive and structured evaluation of device usability not only focusing on effectiveness but also efficiency and satisfaction is required, yet often falls short in existing literature. Mixed methods evaluations, i.e., assessing a combination of quantitative and qualitative measures, allow to obtain a more holistic picture of all relevant aspects of device usability. Considering these aspects already in early development stages allows to identify design issues and generate generalizable benchmarks for future developments.

METHODS

We evaluated the short-term usability of the RELab tenoexo, a RHO for hand function assistance, in 15 users with tetraplegia after a spinal cord injury through a comprehensive mixed methods approach. We collected quantitative data using the Action Research Arm Test (ARAT), the System Usability Scale (SUS), and timed tasks such as the donning process. In addition, qualitative data were collected through semi-structured interviews and user observations, and analyzed with a thematic analysis to enhance the usability evaluation. All insights were attributed and discussed in relation to specifically defined usability attributes such as comfort, ease of use, functional benefit, and safety.

RESULTS

The RELab tenoexo provided an immediate functional benefit to the users, resulting in a mean improvement of the ARAT score by 5.8 points and peaking at 15 points improvement for one user (clinically important difference: 5.7 points). The mean SUS rating of 60.6 represents an adequate usability, however, indicating that especially the RHO donning (average task time = 295 s) was perceived as too long and cumbersome. The participants were generally very satisfied with the ergonomics (size, dimensions, fit) of the RHO. Enhancing the ease of use, specifically in donning, increasing the provided grasping force, as well as the availability of tailoring options and customization were identified as main improvement areas to promote RHO usability.

CONCLUSION

The short-term usability of the RELab tenoexo was thoroughly evaluated with a mixed methods approach, which generated valuable data to improve the RHO in future iterations. In addition, learnings that might be transferable to the evaluation and design of other RHO were generated, which have the potential to increase the daily life applicability and acceptance of similar technologies.

A qualitative study to elicit user requirements for lower limb wearable exoskeletons for gait rehabilitation in spinal cord injury

OBJECTIVE

We aim to determine a comprehensive set of requirements, perceptions, and expectations that people with spinal cord injury (SCI) and the clinicians in charge of their rehabilitation have regarding the use of wearable robots (WR) for gait rehabilitation.

BACKGROUND

There are concerns due to the limited user acceptance of WR for gait rehabilitation. Developers need to emphasize understanding the needs and constraints of all stakeholders involved, including the real-life dynamics of rehabilitation centers.

METHODS

15 people with SCI, 9 without experience with WR and 6 with experience with these technologies, and 10 clinicians from 3 rehabilitation centers in Spain were interviewed. A directed content analysis approach was used.

RESULTS

78 codes grouped into 9 categories (physical results, usability, psychology-related codes, technical characteristics, activities, acquisition issues, context of use, development of the technologies and clinical rehabilitation context) were expressed by at least 20% of the users interviewed, of whom 16 were not found in the literature. The agreement percentage between each group and subgroup included in the study, calculated as the number of codes that more than 20% of both groups expressed, divided over the total amount of codes any of those two groups agreed on (≥ 20%), showed limited agreement between patients and clinicians (50.00%) and between both types of patients (55.77%). The limited accessibility and availability of lower limb exoskeletons for gait rehabilitation arose in most of the interviews.

CONCLUSIONS

The limited agreement percentage between patients and clinicians indicates that including both types of users in the design process of these technologies is important, given that their requirements are complementary. Engaging users with prior technology experience is recommended, as they often exhibit strong internal consensus and articulate well-defined requirements. This study adds up the knowledge available in the literature and the new codes found in our data, which enlighten important aspects that ought to be addressed in the field to develop technologies that respond to users' needs, are usable and feasible to implement in their intended contexts.

APPLICATION

The set of criteria summarized in our study will be useful to guide the design, development, and evaluation of WR for gait rehabilitation to meet user's needs and allow them to be implemented in their intended context of use.