Agile, Easily Applicable, and Useful eHealth Usability Evaluations: Systematic Review and Expert-Validation

Leagility in the healthcare research: a systematic review

BACKGROUND

Expenditure of healthcare services has been growing over the past decades. Lean and agile are two popular paradigms that could potentially contain cost and improve proficiency of the healthcare system. However no systematic review was found on leagilty in the healthcare research. This study aims at synthesizing the extant literature of leagility in the healthcare area to consolidate its potential and identify research gaps for future study in the field.

METHODS

A systematic literature review is conducted following the PRISMA checklist approach. Studies were searched in multiple databases. The selection of articles was executed by dual-scanning of two researchers to ensure quality of data and relevance to the topic. Scientific articles published between January 1999 and November 2023 concerning leagile healthcare are analysed using Microsoft Excel and VOSviewer (version 1.6.18).

RESULTS

Out of 270 articles identified from the inclusion and exclusion criteria, 24 were included in the review. A total of 11 target areas were identified in leagility applications in healthcare. Success and limiting factors of leagile healthcare were classified into macro and micro aspects and further categorized into six dimensions: policy, organization, human resources, marketing, operation management and technology. Moreover, four research gaps were revealed and suggestions were provided for future study.

CONCLUSION

Leagility in the healthcare context is still being in its infancy. Few empirical validation was found in leagile healthcare literature. Further exploration into the application of theory in various sectors under the scope of healthcare is appealed for. Standardization and modularization, leadership support, skillfulness of professionals and staff training are the factors most frequently mentioned for a successful implementation of leagility in the healthcare sector.

An Online Psychological Program for Adolescents and Young Adults With Headaches: Iterative Design and Rapid Usability Testing

BACKGROUND

Headache disorders are common, debilitating health problems. Cognitive-behavioral therapy (CBT) is recommended but rarely easily available. With the use of the internet and communication technologies among youth and young adults, these individuals could be self-trained in CBT skills. There is an increasing number of internet-based interventions for headaches, but there has been little research into the usability of these interventions because evaluating usability across the intervention development life cycle is costly. We developed an internet-based CBT program, the Specialized Program for Headache Reduction (SPHERE). While developing it, we aimed to improve SPHERE through rapid usability testing cycles.

OBJECTIVE

This study aims to presents a rapid and affordable usability testing approach that can be performed throughout the intervention development life cycle. This paper also provides evidence of the usability of SPHERE.

METHODS

We used the "think aloud" usability testing method based on Krug's approach to test user interaction within a lab setting. This was followed by a short posttest interview. We planned to test SPHERE with 3-5 participants testing the same part of the program each cycle. Both the design and development team and the research team actively participated in the usability testing process. Observers independently identified the top 3 usability issues, rated their severity, and conducted debriefing sessions to come to consensus on major issues and generate potential solutions.

RESULTS

The testing process allowed major usability issues to be identified and rectified rapidly before piloting SPHERE in a real-world context. A total of 2 cycles of testing were conducted. Of the usability issues encountered in cycles 1 and 2, a total of 68% (17/25) and 32% (12/38), respectively, were rated as major, discussed, and fixed.

CONCLUSIONS

This study shows that rapid usability testing is an essential part of the design process that improves program functionality and can be easy and inexpensive to undertake.

The "Breast-4Y" web app for breast cancer prevention at young age: Development, evaluation, and validation

BACKGROUND

Improving knowledge of risk factors, signs and symptoms positively influence an individual's intention to acquire healthy lifestyle behaviors to prevent breast cancer, to identify breast cancer risk early and seek health advice early. The aim of the present study was to develop and to assess the usability and quality of a web-app (Breast-4Y) designed specifically for younger women to improve their knowledge about breast cancer risk factors and symptoms as well as protective behaviors to reduce their risk.

METHODS

The development of the Breast-4Y web-app, based on Health Belief Model and the Behaviour Change Wheel, comprised four steps: i) analysis of scientific literature regarding breast cancer risk factors, symptoms and preventive lifestyle behaviors; ii) design of web-app; iii) content review by experts; iv) quality and usability assessment by the end-users (n = 20). Web-app usability was evaluated using the Spanish version of the System Usability Scale for the Assessment of Electronic Tools and the quality using the Spanish version of uMARS.

RESULTS

The contents were rated 4.25 or above, indicating high content quality. The mean usability score was 84.9 (SD 12.4), the mean rating for objective quality and subjective quality dimensions mean were 4.2 (SD 0.4) and 3.4 (0.777) respectively, indicating excellent usability and quality. Positive associations (p < .05) were observed between numbers of app used and attitudes to change (r = 0.479), and intention to change (r = 0.539). Strong positive associations (p < .001) were also observed between usability satisfaction and attitudes to change (r = 0.584), intention to change (r = 0.656), and help-seeking (r = 0.656).

CONCLUSIONS

This study provided evidence that Breast-4Y has adequate contents, high quality, and usability. Breast-4Y can be tested in pragmatic trials to assess their effectiveness to reduce the risk of breast cancer, raise women's awareness of breast cancer risk factors, increase women's knowledge of breast cancer symptoms, and adopt protective lifestyle behaviors.

Improving the User Experience with Discount Site-Specific User Testing

OBJECTIVES

Poor electronic health record (EHR) usability is associated with patient safety concerns, user dissatisfaction, and provider burnout. EHR certification requires vendors to perform user testing. However, there are no such requirements for site-specific implementations. Health care organizations customize EHR implementations, potentially introducing usability problems. Site-specific usability evaluations may help to identify these concerns, and "discount" usability methods afford health systems a means of doing so even without dedicated usability specialists. This report characterizes a site-specific discount user testing program launched at an academic medical center. We describe lessons learned and highlight three of the EHR features in detail to demonstrate the impact of testing on implementation decisions and on users.

METHODS

Thirteen new EHR features which had already undergone heuristic evaluation and iterative design were evaluated over the course of three user test events. Each event included five to six users. Participants used think aloud technique. Measures of user efficiency, effectiveness, and satisfaction were collected. Usability concerns were characterized by the type of usability heuristic violated and by correctability.

RESULTS

Usability concerns occurred at a rate of 2.5 per feature tested. Seventy percent of the usability concerns were deemed correctable prior to implementation. The first highlighted feature was moved to production despite low single ease question (SEQ) scores which may have predicted its subsequent withdrawal from production based on post implementation feedback. Another feature was rebuilt based on usability findings, and a new version was retested and moved to production. A third feature highlights an easily correctable usability concern identified in user testing. Quantitative usability metrics generally reinforced qualitative findings.

CONCLUSION

Simplified user testing with a limited number of participants identifies correctable usability concerns, even after heuristic evaluation. Our discount usability approach to site-specific usability has a role in implementations and may improve the usability of the EHR for the end user.