Usability: An introduction to and literature review of usability testing for educational resources in radiation oncology

From Treatment to Recovery: Gynecological Survivors' and Caregivers' Perspectives About the Usability of an Educational Resource

The objective of this study was to understand gynecological cancer (GC) survivors' and their informal caregivers' perceptions about the usability of an educational resource to support their transition from primary cancer treatment into surveillance and/or recovery. After developing an empirical- and experiential-informed educational resource, we used a semi-structured questioning process to understand GC survivors and their caregivers' perceptions about its usability. Data were collected via online focus groups or 1:1 interviews that were audio recorded and transcribed. We used thematic analysis to analyze the data. Ten participants who were survivors or informal caregivers of cervical, ovarian, or uterine/endometrial cancer participated in two rounds of data collection. We grouped qualitative data into two themes: (1) reputable, relevant, and accessible education reduces uncertainty and promotes connection, and (2) individualized delivery of education provided by trusted cancer clinicians. The transition from treatment to surveillance is a challenging time for which reputable, relevant, and accessible educational resources are useful to facilitate an understanding about and self-management of survivorship-related concerns. Survivors and caregivers look to clinicians to provide reputable education to address their needs. This education should be diverse in content and referred to repeatedly throughout the cancer trajectory.

Assessing the usability of Accessercise to increase physical activity in adults with physical disabilities: A qualitative think-aloud study

BACKGROUND

Mobile health (mHealth) applications (apps) offer a convenient way to increase physical activity to people with disabilities. While several mHealth apps have been developed for this population, there is limited evidence assessing app usability and how this might impact physical activity.

OBJECTIVE

To investigate the usability of a novel mHealth app, Accessercise, that aims to increase physical activity in people with physical disabilities.

METHODS

Twelve adults with different physical disabilities participated in a face-to-face think-aloud interview. Interviews were analysed using deductive content analysis based on the User Version of the Mobile Application Rating Scale (uMARS).

FINDINGS

Data mapped onto 12 uMARS domains; most participants expressed positive views concerning Accessercise, namely, entertainment, customisation, tailoring to the target group, ease of use and navigation, and visual information. Some additional elements were viewed positively but required modification to improve usability, perceived credibility, and relevance, including the app's layout, visual appeal, interactivity, and quality/quantity of information.

CONCLUSION

This study provides an example of how the think-aloud method can be employed to evaluate mHealth apps that aims to increase physical activity in people with physical disabilities. Therefore, developers and researchers could use this study to inform future usability evaluations in this area.

Perspectives on usability and adoption of a new ACL injury prevention programme for female handball players: a mixed methods approach

Effective injury prevention programmes (IPPs) hold promise for mitigating ACL injuries in female handball players, yet adherence remains a challenge. This mixed methods study explores the relationship between programme usability characteristics and the potential effectiveness of a newly developed ACL IPP through survey and interview data from 23 female handball players aged 15-18 and their four coaches. Players' knee abduction moment (KAM) was measured during a cutting task and used to measure potential effectiveness. Findings reveal significant correlations between reductions in KAM and players' perceived effectiveness of the programme in reducing ACL injury risk and their intention to use it (Spearman's rho (rs) -0.52, 95% CI -0.78 to -0.1, p=0.02 and rs -0.46, 95% CI -0.75 to -0.03, p=0.04, respectively). Coaches and players identified efficiency and perceived effectiveness as key factors influencing programme adoption. Concerns regarding programme length and doubts about exercise efficacy and transferability emerged as barriers, while perceived effectiveness, efficiency and enjoyability were facilitators for programme use. Players' suggestions for programme improvement included shortening the duration and incorporating playful elements. Design thinking sessions yielded strategies to optimise time efficiency and integrate IPPs into existing training routines. The study reaffirms the importance of balancing programme effectiveness with practicality and clear communication about its purpose and benefits to facilitate programme adoption. Enhancing programme usability and involving all end-users in the design process are crucial steps toward promoting the acceptance and adoption of IPPs among target populations.

Usability testing of a conceptual model through retrospective cross-case analysis

Qualitative conceptual models are commonly used in the scientific literature to make complex phenomena easier to understand. However, the effectiveness and usability of conceptual models to serve as analytical tools is rarely explored and tested, and there is a lack of guidelines for such analyses. This paper adapts and combines the methods of usability testing and cross-case analysis to describe a systematic protocol to facilitate the qualitative evaluation of conceptual models. Usability testing is an established method for identifying problems or shortcomings within a product and for assessing different dimensions of product usability: suitability, accessibility, relevance, and integrity. Cross-case analysis, on the other hand, is a qualitative research method for systematically comparing information from individual case studies and identifying commonalities and patterns that apply across cases. Taken together, these methodological approaches provide a structured way of retrospectively applying a conceptual model to existing literature and thereby evaluating its effectiveness in meeting its intended purpose.•We show how researchers can prepare, conduct, and synthesise the results of a usability test of a conceptual model.•We provide recommendations for the practical implementation of each step.•We outline the benefits, limitations, and ethical considerations that researchers should be aware of.

Development and Validation of a Treatment Algorithm for Osteoarthritis Pain Management in Patients With End-Stage Kidney Disease Undergoing Hemodialysis

Background

Although osteoarthritis is common in the hemodialysis population and leads to poor health outcomes, pain management is challenged by the absence of clinical guidance. A treatment algorithm was developed and validated to aid hemodialysis clinicians in managing osteoarthritis pain.

Objective

The objective was to develop and validate a treatment algorithm for managing osteoarthritis pain in patients undergoing hemodialysis.

Design

A validation study was conducted based on Lynn's method for content validation.

Setting

To develop and validate a treatment algorithm, interviews were conducted virtually by the primary researcher with clinicians from various institutions across the Greater Toronto and Hamilton Area in Ontario.

Patients

The treatment algorithm was developed and validated for the management of osteoarthritis pain in patients on hemodialysis. Patients were not involved in the development or validation of the tool.

Measurements

The algorithm was measured for content and face validity. Content validity was measured by calculating the content validity index of each component (I-CVI) of the algorithm and the overall scale validity index (S-CVI). Face validity was assessed by calculating the percentage of positive responses to the face validity statements.

Methods

A draft algorithm was developed based on literature searches and expert opinion and validated by interviewing nephrology and pain management clinicians. Through consecutive rounds of 1:1 interviews, content and face validity were assessed by asking participants to rate the relevance of each component of the algorithm and indicate their level of agreeability with a series of statements. Following each round, the I-CVI of the algorithm as well as the S-CVI was calculated and the percentage of positive responses to the statements was determined. The research team revised the algorithm in response to the findings. The final algorithm provides a stepwise approach to the non-pharmacologic and pharmacologic management of pain, including topical, oral, and opioid use.

Results

A total of 18 clinicians from 7 institutions across the Greater Toronto and Hamilton Area were interviewed (10 pharmacists, 5 nurse practitioners, and 3 physicians). The average S-CVI of the algorithm across all 3 rounds was 0.93. At least 78% of participants provided positive responses to the face validity statements.

Limitations

An algorithm was developed based on input from clinicians working in the province of Ontario, limiting the generalizability of the algorithm across provinces. In addition, the algorithm did not include the perspectives of primary care providers or patients/caregivers.

Conclusions

An algorithm for the management of osteoarthritis pain in the hemodialysis population was developed and validated through expert review to standardize practices and encourage clinicians to use evidence-based treatments and address the psychosocial symptoms of pain. As the algorithm possesses a high degree of content and face validity, it may improve osteoarthritis pain management among patients undergoing hemodialysis. Future research will assess the implementation of the algorithm across hemodialysis settings.

CheckPOINT: a simple tool to measure Surgical Safety Checklist implementation fidelity

INTRODUCTION

The WHO Surgical Safety Checklist (SSC) is a communication tool that improves teamwork and patient outcomes. SSC effectiveness is dependent on implementation fidelity. Administrative audits fail to capture most aspects of SSC implementation fidelity (ie, team communication and engagement). Existing research tools assess behaviours during checklist performance, but were not designed for routine quality assurance and improvement. We aimed to create a simple tool to assess SSC implementation fidelity, and to test its reliability using video simulations, and usability in clinical practice.

METHODS

The Checklist Performance Observation for Improvement (CheckPOINT) tool underwent two rounds of face validity testing with surgical safety experts, clinicians and quality improvement specialists. Four categories were developed: checklist adherence, communication effectiveness, attitude and engagement. We created a 90 min training programme, and four trained raters independently scored 37 video simulations using the tool. We calculated intraclass correlation coefficients (ICC) to assess inter-rater reliability (ICC>0.75 indicating excellent reliability). We then trained two observers, who tested the tool in the operating room. We interviewed the observers to determine tool usability.

RESULTS

The CheckPOINT tool had excellent inter-rater reliability across SSC phases. The ICC was 0.83 (95% CI 0.67 to 0.98) for the sign-in, 0.77 (95% CI 0.63 to 0.92) for the time-out and 0.79 (95% CI 0.59 to 0.99) for the sign-out. During field testing, observers reported CheckPOINT was easy to use. In 98 operating room observations, the total median (IQR) score was 25 (23-28), checklist adherence was 7 (6-7), communication effectiveness was 6 (6-7), attitude was 6 (6-7) and engagement was 6 (5-7).

CONCLUSIONS

CheckPOINT is a simple and reliable tool to assess SSC implementation fidelity and identify areas of focus for improvement efforts. Although CheckPOINT would benefit from further testing, it offers a low-resource alternative to existing research tools and captures elements of adherence and team behaviours.

Adapting TeachABI to the local needs of Australian educators - a critical step for successful implementation

Background The present study is the foundational project of TeachABI-Australia , which aims to develop and implement an accessible, nation-wide digital resource for educators to address their unmet acquired brain injury (ABI)-related professional learning needs. The aim of the present study was to identify the adaptations required to improve the suitability and acceptability of the TeachABI professional development module within the Australian education system from the perspectives of Australian educators. Methods The research design employed an integrated knowledge translation approach and followed the ADAPT Guidance for undertaking adaptability research. A purposive sample of eight educators eligible to teach primary school in Australia provided feedback on the module through a quantitative post-module feedback questionnaire and a qualitative semi-structured interview. Results Participants rated the acceptability of the module as 'Completely Acceptable ' (Mdn  = 5, IQR = 1), and reported 'only Minor' changes were required (Mdn  = 2, IQR = 0.25) to improve the suitability to the Australian context. Qualitative analysis of transcripts revealed three broad categories: (1) the usefulness of TeachABI , (2) the local fit of TeachABI , and (3) pathways for implementing TeachABI in the local setting. Recommended adaptations to the module collated from participant feedback included changes to language, expansion of content, and inclusion of Australian resources, legislation, and videos. Conclusions TeachABI is acceptable to Australian educators but requires modifications to tailor the resource to align with the unique schooling systems, needs, and culture of the local setting. The systematic methodological approach to adaptation outlined in this study will serve as a guide for future international iterations of TeachABI .