Methods of usability testing in the development of eHealth applications: A scoping review

A Technological Tool Aimed at Self-Care in Patients With Multimorbidity: Cross-Sectional Usability Study

BACKGROUND

Information and communication technologies (ICTs) have been positioned as useful tools to facilitate self-care. The interaction between a patient and technology, known as usability, is particularly important for achieving positive health outcomes. Specific characteristics of patients with chronic diseases, including multimorbidity, can affect their interaction with different technologies. Thus, studying the usability of ICTs in the field of multimorbidity has become a key element to ensure their relevant role in promoting self-care.

OBJECTIVE

The aim of this study was to analyze the usability of a technological tool dedicated to health and self-care in patients with multimorbidity in primary care.

METHODS

A descriptive observational cross-sectional usability study was performed framed in the clinical trial in the primary care health centers of Madrid Health Service of the TeNDER (Affective Based Integrated Care for Better Quality of Life) project. The TeNDER technological tool integrates sensors for monitoring physical and sleep activity along with a mobile app for consulting the data collected and working with self-management tools. This project included patients over 60 years of age who had one or more chronic diseases, at least one of which was mild-moderate cognitive impairment, Parkinson disease, or cardiovascular disease. From the 250 patients included in the project, 38 agreed to participate in the usability study. The usability variables investigated were effectiveness, which was determined by the degree of completion and the total number of errors per task; efficiency, evaluated as the average time to perform each task; and satisfaction, quantified by the System Usability Scale. Five tasks were evaluated based on real case scenarios. Usability variables were analyzed according to the sociodemographic and clinical characteristics of patients. A logistic regression model was constructed to estimate the factors associated with the type of support provided for task completion.

RESULTS

The median age of the 38 participants was 75 (IQR 72.0-79.0) years. There was a slight majority of women (20/38, 52.6%) and the participants had a median of 8 (IQR 7.0-11.0) chronic diseases. Thirty patients completed the usability study, with a usability effectiveness result of 89.3% (134/150 tasks completed). Among the 30 patients, 66.7% (n=20) completed all tasks and 56.7% (17/30) required personalized help on at least one task. In the multivariate analysis, educational level emerged as a facilitating factor for independent task completion (odds ratio 1.79, 95% CI 0.47-6.83). The median time to complete the total tasks was 296 seconds (IQR 210.0-397.0) and the median satisfaction score was 55 (IQR 45.0-62.5) out of 100.

CONCLUSIONS

Although usability effectiveness was high, the poor efficiency and usability satisfaction scores suggest that there are other factors that may interfere with the results. Multimorbidity was not confirmed to be a key factor affecting the usability of the technological tool.

TRIAL REGISTRATION

Clinicaltrials.gov NCT05681065; https://clinicaltrials.gov/study/NCT05681065.

Designing for usability: development and evaluation of a portable minimally-actuated haptic hand and forearm trainer for unsupervised stroke rehabilitation

In stroke rehabilitation, simple robotic devices hold the potential to increase the training dosage in group therapies and to enable continued therapy at home after hospital discharge. However, we identified a lack of portable and cost-effective devices that not only focus on improving motor functions but also address sensory deficits. Thus, we designed a minimally-actuated hand training device that incorporates active grasping movements and passive pronosupination, complemented by a rehabilitative game with meaningful haptic feedback. Following a human-centered design approach, we conducted a usability study with 13 healthy participants, including three therapists. In a simulated unsupervised environment, the naive participants had to set up and use the device based on written instructions. Our mixed-methods approach included quantitative data from performance metrics, standardized questionnaires, and eye tracking, alongside qualitative feedback from semi-structured interviews. The study results highlighted the device's overall ease of setup and use, as well as its realistic haptic feedback. The eye-tracking analysis further suggested that participants felt safe during usage. Moreover, the study provided crucial insights for future improvements such as a more intuitive and comfortable wrist fixation, more natural pronosupination movements, and easier-to-follow instructions. Our research underscores the importance of continuous testing in the development process and offers significant contributions to the design of user-friendly, unsupervised neurorehabilitation technologies to improve sensorimotor stroke rehabilitation.

Evaluating the Usability of an mHealth App for Empowering Cancer Survivors With Disabilities: Heuristic Evaluation and Usability Testing

BACKGROUND

More than 18 million cancer survivors are living in the United States. The effects of cancer and its treatments can have cognitive, psychological, physical, and social consequences that many survivors find incredibly disabling. Posttreatment support is often unavailable or underused, especially for survivors living with disabilities. This leaves them to deal with new obstacles and struggles on their own, oftentimes feeling lost during this transition. Mobile health (mHealth) interventions have been shown to effectively aid cancer survivors in dealing with many of the aftereffects of cancer and its treatments; these interventions hold immense potential for survivors living with disabilities. We developed a prototype for WeCanManage, an mHealth-delivered self-management intervention to empower cancer survivors living with disabilities through problem-solving, mindfulness, and self-advocacy training.

OBJECTIVE

Our study conducted a heuristic evaluation of the WeCanManage high-fidelity prototype and assessed its usability among cancer survivors with known disabilities.

METHODS

We evaluated the prototype using Nielsen's 10 principles of heuristic evaluation with 22 human-computer interaction university students. On the basis of the heuristic evaluation findings, we modified the prototype and conducted usability testing on 10 cancer survivors with a variety of known disabilities, examining effectiveness, efficiency, usability, and satisfaction, including a completion of the modified System Usability Scale (SUS).

RESULTS

The findings from the heuristic evaluation were mostly favorable, highlighting the need for a help guide, addressing accessibility concerns, and enhancing the navigation experience. After usability testing, the average SUS score was 81, indicating a good-excellent design. The participants in the usability testing sample expressed positive reactions toward the app's design, educational content and videos, and the available means of connecting with others. They identified areas for improvement, such as improving accessibility, simplifying navigation within the community forums, and providing a more convenient method to access the help guide.

CONCLUSIONS

Overall, usability testing showed positive results for the design of WeCanManage. The course content and features helped participants feel heard, understood, and less alone.

Feasibility, accuracy, and usability analysis of MapAML, a first-in-class app for integrated diagnosis in acute myeloid leukemia

Performing a comprehensive diagnosis of acute myeloid leukemia (AML) is complex and involves the integration of clinical information, bone marrow morphology, immunophenotyping, cytogenetic, and molecular analysis, which can be challenging to the general hematologist. The aim of this study was to evaluate the usability and accuracy of MapAML, a smartphone app for integrated diagnosis in AML, created to aid the hematologist in its clinical practice. App performance was evaluated in dedicated sessions, in which 21 hematologists or fellows in hematology performed an integrated diagnosis of deidentified real-world clinical AML cases, first without and posteriorly with MapAML use. Diagnosis accuracy increased after MapAML utilization, with the average score going from 7.08 without app to 8.88 with app use (on a scale from 0 to 10), representing a significant accuracy improvement (p = .002). Usability evaluation was very favorable, with 81% of users considering the app very or extremely simple to use. There was also a significant increase in confidence to perform a complete and accurate diagnosis in AML after app use, with 61.9% of the participants willing to use the app in their clinical practice. In this study, MapAML increased accuracy with excellent usability for integrated diagnosis in AML.

Comparison of evaluation methods for improving the usability of a Spanish mHealth tool

OBJECTIVE

Mobile health (mHealth) technology is now widely used across health conditions and populations. The rigorous development of these tools has yielded improved health outcomes, yet the ideal approach for developing mHealth tools continues to evolve, indicating the need for rigorous usability evaluation methods. This study compares two usability evaluation methods - cognitive interviews and usability assessments employing a think-aloud approach - for adapting an evidence-based mHealth tool from English into Spanish.

METHODS

We conducted cognitive interviews and usability assessments using a think-aloud protocol to evaluate the usability of an HIV mHealth application among 40 Spanish-speaking adults with HIV in New York City, NY, and La Romana, Dominican Republic. The Health IT Usability Evaluation Model (Health-ITUEM) was used to guide the analysis of qualitative data collected from each method.

RESULTS

Participants (N = 40) averaged 43 years old (SD = 12.26; range 20-79), identified primarily Hispanic/Latino (92.5 %), and resided in La Romana (50 %) or New York City (50 %). Both usability evaluation methods yielded similar findings, highlighting learnability and information needs as crucial components of participant feedback for the mHealth application. Cognitive interviews captured participants' perspectives on the app's interface and design. On the other hand, results from usability assessments offered insights into participants' competency while interacting with the mHealth tool.

CONCLUSION

Findings from this study highlight the contributions and limitations of including cognitive interviews and task-based usability assessments using a think-aloud approach in mHealth usability testing. Future research should employ a multi-method approach, incorporating complementary usability evaluation methods and engaging participants in multiple assessments. Using complementary usability evaluation methods may provide a more comprehensive understanding of the usability and participant experience aspects of a mHealth tool compared to using a single usability evaluation method.

Development of oral health resources and a mobile app for caregivers and autistic children through consensus building

LAY ABSTRACT

Caregivers of autistic children often lack knowledge regarding oral homecare and when and where to see the dentist. To address this need, we developed a series of information on oral health. An autistic child assisted in developing two social stories to showcase a dental visit. A mobile app was developed to deliver the above mentioned. Other features include creation of customised social stories and visual schedule and an inbox to allow dentists to send messages to parents. The developed information and social stories were reviewed by experts and parents. The app also underwent anonymous and independent testing by parents. Overall the information and app were well received by the experts and parents.

World of sounds (Seslerin Dünyası): A mobile auditory training game for children with cochlear implants

OBJECTIVES

The aim of this study is to develop a mobile auditory training application based on gaming for children aged 3-5 years using cochlear implants and to evaluate its usability.

METHODS

4 games were developed in the scope of the application World of Sounds; the crucible sound for auditory awareness, mole hunting for auditory discrimination, find the sound for auditory recognition, and choo-choo for auditory comprehension. The prototype was applied to 20 children with normal hearing and 20 children with cochlear implants, all of whom were aged 3-5. The participants were asked to fill out the Game Evaluation Form for Children. Moreover, 40 parents were included in the study, and the Evaluation Form for the Application was applied.

RESULTS

According to the form, at least 80% of children using cochlear implants, and all children in the healthy group, responded well to the usability factors. All factors were obtained as highly useable by parents of the children using cochlear implants. The results indicated that in the healthy group, the usefulness and motivation factors were above moderate, and the other factors were highly useable. In the mole-hunting game, there was no significant difference between the groups in the easy level of the first sub-section (p > 0.05). There was a significant difference between the groups in terms of the other sub-sections of the mole-hunting game and all sub-sections of the crucible sound, find the sound, and the choo-choo games (p < 0.05). While there was no correlation between duration of cochlear implant use and ADSI scores and the third sub-section of the crucible sound game (p > 0.05); a correlation was found in the other sub-sections of crucible sound and all sub-sections of the mole hunting, find the sound, and Choo-Choo games (p < 0.05).

CONCLUSION

It is thought that the application World of Sounds can serve as an accessible option to support traditional auditory rehabilitation for children with cochlear implants.

WeChat mini program in laboratory biosafety education among medical students at Guangzhou Medical University: a mixed method study of feasibility and usability

BACKGROUND

Laboratory biosafety should be a priority in all healthcare institutions. In traditional laboratory safety teaching students typically receive knowledge passively from their teachers without active involvement. The combination of experiential learning and mobile learning may provide students with greater engagement, retention, and application of knowledge. To address this issue, we developed and conducted a convergent mixed methods study to assess the feasibility and usability of a WeChat mini program (WMP) named WeMed for laboratory biosafety education for medical laboratory students at Guangzhou Medical University (GMU).

METHODS

The study was conducted between November 2022 and October 2023 among second-year undergraduate students at GMU. It involved the concurrent collection, analysis, and interpretation of both qualitative and quantitative data to assess feasibility and usability. In the quantitative strand, two evaluations were conducted via online surveys from students (n = 67) after a four-week study period. The System Usability Scale (SUS) was used to evaluate usability, while self-developed questions were used to assess feasibility. Additionally, a knowledge test was administered 6 months after the program completion. In the qualitative strand, fourteen semi-structured interviews were conducted, whereby a reflexive thematic analysis was utilized to analyze the interview data.

RESULTS

The overall SUS score is adequate (M = 68.17, SD = 14.39). The acceptability of the WeMed program is in the marginal high range. Most students agreed that WeMed was useful for learning biosafety knowledge and skills (13/14, 93%), while 79% (11/14) agreed it was easy to use and they intended to continue using it. After 6 months, a significant difference in the knowledge test scores was observed between the WeMed group (n = 67; 2nd year students) and the traditional training group (n = 90; 3rd year students). However, the results should be interpreted cautiously due to the absence of a pretest.

CONCLUSION

The combination of experiential learning and mobile learning with WMP is a feasible tool for providing laboratory biosafety knowledge and skills. Ongoing improvements should be made in order to increase long-term acceptance.

Testing Two Online Symptom Checkers With Vulnerable Groups: Usability Study to Improve Cognitive Accessibility of eHealth Services

BACKGROUND

The popularity of eHealth services has surged significantly, underscoring the importance of ensuring their usability and accessibility for users with diverse needs, characteristics, and capabilities. These services can pose cognitive demands, especially for individuals who are unwell, fatigued, or experiencing distress. Additionally, numerous potentially vulnerable groups, including older adults, are susceptible to digital exclusion and may encounter cognitive limitations related to perception, attention, memory, and language comprehension. Regrettably, many studies overlook the preferences and needs of user groups likely to encounter challenges associated with these cognitive aspects.

OBJECTIVE

This study primarily aims to gain a deeper understanding of cognitive accessibility in the practical context of eHealth services. Additionally, we aimed to identify the specific challenges that vulnerable groups encounter when using eHealth services and determine key considerations for testing these services with such groups.

METHODS

As a case study of eHealth services, we conducted qualitative usability testing on 2 online symptom checkers used in Finnish public primary care. A total of 13 participants from 3 distinct groups participated in the study: older adults, individuals with mild intellectual disabilities, and nonnative Finnish speakers. The primary research methods used were the thinking-aloud method, questionnaires, and semistructured interviews.

RESULTS

We found that potentially vulnerable groups encountered numerous issues with the tested services, with similar problems observed across all 3 groups. Specifically, clarity and the use of terminology posed significant challenges. The services overwhelmed users with excessive information and choices, while the terminology consisted of numerous complex medical terms that were difficult to understand. When conducting tests with vulnerable groups, it is crucial to carefully plan the sessions to avoid being overly lengthy, as these users often require more time to complete tasks. Additionally, testing with vulnerable groups proved to be quite efficient, with results likely to benefit a wider audience as well.

CONCLUSIONS

Based on the findings of this study, it is evident that older adults, individuals with mild intellectual disability, and nonnative speakers may encounter cognitive challenges when using eHealth services, which can impede or slow down their use and make the services more difficult to navigate. In the worst-case scenario, these challenges may lead to errors in using the services. We recommend expanding the scope of testing to include a broader range of eHealth services with vulnerable groups, incorporating users with diverse characteristics and capabilities who are likely to encounter difficulties in cognitive accessibility.

Digital clinical empathy in a live chat: multiple findings from a formative qualitative study and usability tests

BACKGROUND

Clinical empathy is considered a crucial element in patient-centered care. The advent of digital technology in healthcare has introduced new dynamics to empathy which needs to be explored in the context of the technology, particularly within the context of written live chats. Given the growing prevalence of written live chats, this study aimed to explore and evaluate techniques of digital clinical empathy within a familial cancer-focused live chat, focusing on how health professionals can (a) understand, (b) communicate, and (c) act upon users' perspectives and emotional states.

METHODS

The study utilized a qualitative approach in two research phases. It examined the expected and implemented techniques and effectiveness of digital clinical empathy in a live chat service, involving semi-structured interviews with health professionals (n = 9), focus group discussions with potential users (n = 42), and two rounds of usability tests between health professionals (n = 9) and users (n = 18). Data were examined using qualitative content analysis.

RESULTS

Expected techniques of digital clinical empathy, as articulated by both users and health professionals, involve reciprocal engagement, timely responses, genuine authenticity, and a balance between professionalism and informality, all while going beyond immediate queries to facilitate informed decision-making. Usability tests confirm these complexities and introduce new challenges, such as balancing timely, authentic responses with effective, personalized information management and carefully framed referrals.

CONCLUSIONS

The study reveals that the digital realm adds layers of complexity to the practice of clinical empathy. It underscores the importance of ongoing adaptation and suggests that future developments could benefit from a hybrid model that integrates the strengths of both AI and human health professionals to meet evolving user needs and maintain high-quality, empathetic healthcare interactions.

Usability and Evaluation of a Health Information System in the Emergency Department: Mixed Methods Study

BACKGROUND

A lack of information during an emergency visit leads to the experience of powerlessness for patients and their family members, who may also feel unprepared to cope with acute symptoms. The ever-changing nature and fast-paced workflow in the emergency department (ED) often affect how health care professionals can tailor information and communication to the needs of the patient.

OBJECTIVE

This study aimed to evaluate the usability and experience of a newly developed information system. The system was developed together with patients and their family members to help provide the information needed in the ED.

METHODS

We conducted a mixed methods study consisting of quantitative data obtained from the System Usability Scale questionnaire and qualitative interview data obtained from purposively selected participants included in the quantitative part of the study.

RESULTS

A total of 106 patients and 14 family members (N=120) answered the questionnaire. A total of 10 patients and 3 family members participated in the interviews. Based on the System Usability Scale score, the information system was rated close to excellent, with a mean score of 83.6 (SD 12.8). Most of the participants found the information system easy to use and would like to use it again. The participants reported that the system helped them feel in control, and the information was useful. Simplifications were needed to improve the user experience for the older individuals.

CONCLUSIONS

This study demonstrates that the usability of the information system is rated close to excellent. It was perceived to be useful as it enabled understanding and predictability of the patient's trajectory in the ED. Areas for improvement include making the system more usable by older individuals. The study provides an example of how a technological solution can be used to diminish the information gap in an ED context.

Web-Based Screening, Brief Intervention, and Referral to Treatment for Traumatic Stress and Alcohol Misuse Among Survivors of Sexual Assault and Intimate Partner Violence: Usability and Acceptability Study

BACKGROUND

Recent survivors of intimate partner violence (IPV) and sexual assault (SA) are at a high risk for traumatic stress and alcohol misuse. IPV and SA survivors face barriers to services for traumatic stress and alcohol misuse and have low service utilization rates. One way to increase access to services for this population is the use of web-based screening, brief intervention, and referral to treatment (SBIRT), an evidence-informed approach for early identification of traumatic stress and alcohol and drug misuse and connecting individuals to treatment.

OBJECTIVE

This study aims to assess the usability and acceptability of a web-based SBIRT called CHAT (Choices For Your Health After Trauma) tailored to address traumatic stress and alcohol misuse following past-year IPV, SA, or both.

METHODS

Phase 1 involved gathering feedback about usability and acceptability from focus groups with victim service professionals (22/52, 42%) and interviews with past-year survivors of IPV, SA, or both (13/52, 25%). Phase 2 involved gathering feedback about the acceptability of an adapted version of CHAT in an additional sample of recent survivors (17/52, 33%). Survey data on history of IPV and SA, posttraumatic stress disorder symptoms, alcohol and drug use, and service use were collected from survivors in both phases to characterize the samples. Qualitative content and thematic analyses of the interviews and focus group data were conducted using a coding template analysis comprising 6 a priori themes (usability, visual design, user engagement, content, therapeutic persuasiveness, and therapeutic alliance).

RESULTS

Six themes emerged during the focus groups and interviews related to CHAT: usability, visual design, user engagement, content, therapeutic persuasiveness, and therapeutic alliance. Phase 1 providers and survivors viewed CHAT as acceptable, easy to understand, and helpful. Participants reported that the intervention could facilitate higher engagement in this population as the web-based modality is anonymous, easily accessible, and brief. Participants offered helpful suggestions for improving CHAT by updating images, increasing content personalization, reducing text, and making users aware that the intervention is confidential. The recommendations of phase 1 participants were incorporated into CHAT. Phase 2 survivors viewed the revised intervention and found it highly acceptable (mean 4.1 out of 5, SD 1.29). A total of 4 themes encapsulated participant's favorite aspects of CHAT: (1) content and features, (2) accessible and easy to use, (3) education, and (4) personalization. Six survivors denied disliking any aspect. The themes on recommended changes included content and features, brevity, personalization, and language access. Participants provided dissemination recommendations.

CONCLUSIONS

Overall, CHAT was acceptable among victim service professionals and survivors. Positive reactions to CHAT show promise for future research investigating the efficacy and potential benefit of CHAT when integrated into services for people with traumatic stress and alcohol misuse after recent IPV and SA.

If we build it, will they come? A scoping review of objective engagement metrics in asynchronous psychosocial telehealth interventions for breast cancer survivors

Burgeoning technologies and the COVID-19 pandemic resulted in a boom of telehealth for immunocompromised patients, such as those with cancer. Telehealth modalities overcome barriers and promote accessibility to care. Currently, efficacious psychosocial interventions exist to address negative aftereffects of a cancer diagnosis and treatment. Many of these interventions often incorporate asynchronous telehealth (e.g., web-based, smartphone mobile app) features. However, asynchronous platforms are limited by suboptimal engagement. Subjective indicators of perceived engagement in the forms of acceptability, feasibility, and adherence are often captured, yet prior research has found discrepancies between perceived and actual engagement. The FITT (frequency, intensity, time/duration, type of engagement) model, originally developed for use to quantify engagement within exercise trials, provides a framework to assess objective engagement of psychosocial interventions for breast cancer. Using 14 keywords and searching six databases through 11/2023, 56 studies that used asynchronous telehealth interventions in breast cancer were identified. All FITT domains were reported at least once across studies with intensity metrics most commonly reported. Nine metrics were described across FITT domains. Human-centered design principles to guide telehealth development and privacy considerations are discussed. Findings offer suggestions for how to represent and optimize objective engagement in asynchronous telehealth cancer care.

Design of mobile and website health application devices for drug tolerability in hereditary fructose intolerance

BACKGROUND

Hereditary fructose intolerance (HFI) is a rare metabolic disease caused by aldolase B deficiency. The aim of our study was to analyse excipient tolerability in patients with HFI and other related diseases and to design mobile and website health applications to facilitate the search for drugs according to their tolerance.

RESULTS

A total of 555 excipients listed in the Spanish Medicines Agency database (July 2023) were classified as suitable for HFI patients, suitable with considerations ((glucose and glucose syrup, intravenous sucrose, oral mannitol, polydextrose, gums and carrageenans, ethanol, sulfite caramel and vanilla), not recommended (intravenous mannitol) and contraindicated (fructose, oral sucrose, invert sugar, sorbitol, maltitol, lactitol, isomaltitol, fruit syrups, honey, sucrose esters and sorbitol esters). Glucose and glucose syrup were classified as suitable with considerations due to its possible fructose content and their potential endogenous fructose production. For other related intolerances, wheat starch was contraindicated and oatmeal was not recommended in celiac disease; oral lactose and lactose-based coprocessed excipient (Cellactose®) were not recommended in lactose intolerance; and glucose, invert sugar and oral sucrose were not recommended in diabetes mellitus. The applications were named IntoMed®. Results are listed in order of tolerability (suitable drugs appear first and contraindicated drugs at the end), and they are accompanied by a note detailing their classified excipients. If a drug contains excipients within different categories, the overall classification will be the most restrictive. The apps are also able to classify substances with the same criteria if they act as active ingredients. The tools exhibited good usability (82.07 ± 13.46 points on the System Usability Scale [range: 0-100]) on a sample of HFI patients, their families and health care professionals.

CONCLUSIONS

IntoMed® is a tool for finding information about the tolerability of drugs according to excipients for patients with HFI and other related intolerances, with good usability. It is a fast and reliable system that covers the current excipient legislation and expands on it with other specific information: HFI patients should be alert for excipients such as mannitol (especially in intravenous drugs), fruit syrups, honey, sulfite caramel or vanilla. Glucose might contain or produce fructose, and special precaution is needed because of potential errors in their composition.

Unlocking biomedical data sharing: A structured approach with digital twins and artificial intelligence (AI) for open health sciences

Objective

Data sharing promotes the scientific progress. However, not all data can be shared freely due to privacy issues. This work is intended to foster FAIR sharing of sensitive data exemplary in the biomedical domain, via an integrated computational approach for utilizing and enriching individual datasets by scientists without coding experience.

Methods

We present an in silico pipeline for openly sharing controlled materials by generating synthetic data. Additionally, it addresses the issue of inexperience to computational methods in a non-IT-affine domain by making use of a cyberinfrastructure that runs and enables sharing of computational notebooks without the need of local software installation. The use of a digital twin based on cancer datasets serves as exemplary use case for making biomedical data openly available. Quantitative and qualitative validation of model output as well as a study on user experience are conducted.

Results

The metadata approach describes generalizable descriptors for computational models, and outlines how to profit from existing data resources for validating computational models. The use of a virtual lab book cooperatively developed using a cloud-based data management and analysis system functions as showcase enabling easy interaction between users. Qualitative testing revealed a necessity for comprehensive guidelines furthering acceptance by various users.

Conclusion

The introduced framework presents an integrated approach for data generation and interpolating incomplete data, promoting Open Science through reproducibility of results and methods. The system can be expanded from the biomedical to any other domain while future studies integrating an enhanced graphical user interface could increase interdisciplinary applicability.

Benchmarking usability of patient portals in Estonia, Finland, Norway, and Sweden

INTRODUCTION

Poor usability is a barrier to widespread adoption of electronic health records (EHR). Providing good usability is especially challenging in the health care context, as there is a wide variety of patient users. Usability benchmarking is an approach for improving usability by evaluating and comparing the strength and weaknesses of systems. The main purpose of this study is to benchmark usability of patient portals across countries.

METHODS

A mixed-methods survey approach was applied to benchmark the national patient portals offering patient access to EHR in Estonia, Finland, Norway, and Sweden. These Nordic countries have similar public healthcare systems, and they are pioneers in offering patients access to EHR for several years. In a survey of 29,334 patients, both patients' quantitative ratings of usability and their qualitative descriptions of very positive and very negative peak experiences of portal use were collected.

RESULTS

The usability scores ranged from good to fair level of usability. The narratives of very positive and very negative experiences included the benefits of the patient portals and experienced usability issues. The regression analysis of results showed that very positive and negative experiences of patient portal use explain 19-35% of the variation of usability scores in the four countries. The percentage of patients who reported very positive or very negative experiences in each country was unrelated to the usability scores across countries.

CONCLUSIONS

The survey approach could be used to evaluate usability with a wide variety of users and it supported learning from comparison across the countries. The combination of quantitative and qualitative data provided an approximation of the level of the perceived usability, and identified usability issues to be improved and useful features that patients appreciate. Further work is needed to improve the comparability of the varied samples across countries.

Development and optimisation of a mobile app (iMPAKT) for improving person-centred practice in healthcare settings: A multi-methods evaluation study

Objective

To develop and optimise an app (iMPAKT) for improving implementation and measurement of person-centred practice in healthcare settings.

Methods

Two iterative rounds of testing were carried out based on cognitive task analysis and qualitative interview methods. The System Usability Scale (SUS) was also used to evaluate the app. Quantitative data on task completion and SUS scores were evaluated descriptively, with thematic analysis performed on qualitative data. The MoSCoW prioritisation system was used to identify key modifications to improve the app.

Results

Twelve participants took part (eight health professionals and four patient and public involvement representatives). Views on design and structure of the app were positive. The majority of the 16 tasks undertaken during the cognitive task analysis were easy to complete. Mean SUS scores were 73.5/100 (SD: 7.9; range = 60-92.5), suggesting good overall usability. For one section of the app that transcribes patients speaking about their experience of care, a non-intuitive user interface and lack of transcription accuracy were identified as key issues influencing usability and acceptability.

Conclusions

Findings from the evaluation were used to inform iterative modifications to further develop and optimise the iMPAKT App. These included improved navigational flow, and implementation of an updated artificial intelligence (AI) based Speech-To-Text software; allowing for more accurate, real-time transcription. Use of such AI-based software represents an interesting area that requires further evaluation. This is particularly apparent in relation to potential for large-scale collection of data on person-centred measures using the iMPAKT App, and for assessing initiatives designed to improve patient experience.

Internet-delivered emotional self-management program for the general population during the COVID-19 pandemic: Usability testing

Introduction

Internet-based self-help interventions have the potential to help people address their emotional needs at relatively low costs. However, if the system does not offer optimal functions, it could reduce end-user adherence and satisfaction with treatment and compromise the effectiveness of the program. This study evaluated the usability of an Internet-based self-help intervention for emotional self-management among the general population of Italy during the COVID-19 pandemic.

Methods

A balanced sex-age sample of 10 individuals who met the inclusion criteria were consecutively recruited online. The think-aloud testing method, the system usability scale and an ad hoc semi-structured interview were used to determine the overall system usability.Quantitative data were analyzed using descriptive statistics and qualitative data were analyzed using thematic analysis.

Results

The participants were mostly satisfied with the usability of the program. However, older users (<45 years) encountered some problems, which took longer, made more mistakes, and needed more help in performing the tasks than their younger counterparts. The analysis of the interviews revealed three central themes: general thoughts about the platform, weaknesses of the platform and difficulties encountered while navigating and completing tasks, and strengths of the platform.

Discussion

Based on the results of this study, important improvements will be made before the RinasciMENTE program is tested under real-world conditions. Conducting usability testing is a crucial step at an early stage of the development process of an Internet-based self-help intervention to identify potential usability problems with the system.

User-centered development of an mHealth app for cardiovascular prevention

Background

Mobile health (mHealth) apps can be used for cardiovascular disease (CVD) prevention. User-centered design, evidence-based content and user testing can be applied to ensure a high level of usability and adequate app access.

Objective

To develop and evaluate an mHealth app (HerzFit) for CVD prevention.

Methods

HerzFit´s development included a user-centered design approach and guideline-based content creation based on the identified requirements of the target group. Beta testing and a preliminary usability evaluation of the HerzFit prototype were performed. For evaluation, German versions of the System Usability Scale (SUS) and the mHealth App Usability Questionnaire (GER-MAUQ) as well as free text feedback were applied.

Results

User-centered design thinking led to the definition of four personas. Based on their requirements, HerzFit enables users to individually assess, monitor, and optimize their cardiovascular risk profile. Users are also provided with a variety of evidence-based information on CVD and their risk factors. The user interface and system design followed the identified functional requirements. Beta-testers provided feedback on the structure and functionality and rated the usability of HerzFit´s prototype as slightly above average both in SUS and GER-MAUQ rating. Participants positively noted the variety of functions and information presented in HerzFit, while negative feedback mostly concerned wearable synchronization.

Conclusions

The present study demonstrates the user-centered development of a guideline-based mHealth app for CVD prevention. Beta-testing and a preliminary usability study were used to further improve the HerzFit app until its official release.

Development of a patient-centered transition program for stroke survivors and their informal caregivers, combining case-management and access to an online information platform: A user-centered design approach

Background

During the hospital-to-home transition, stroke survivors and their caregivers face a significant lack of support and information which impacts their psychosocial recovery. We aimed to co-design a program combining individual support by a trained case-manager (dedicated professional providing individual support) and an online information platform to address needs of stroke survivors and caregivers.

Methods

A two-step methodology was used. The first step followed a "user-centered design" approach during four workshops with stroke survivors, caregivers, and healthcare professionals to develop the platform and define the case-manager profile. The second step was a usability test of the platform following a Think Aloud method with patients and caregivers. The workshops and interviews were analyzed following a qualitative thematic analysis. The analysis of Think Aloud interviews was based on User Experience Honeycomb framework by Morville.

Results

Eight participants attended the workshops: two patients, two caregivers, three nurses, and a general practitioner. Activities, training, and skills of the case-manager were defined according to stroke survivors and caregivers needs. Name, graphics, navigation, and content of the platform were developed with the participants, a developer and a graphic designer. The usability of the platform was tested with 5 patients and 5 caregivers. The Think Aloud confirmed satisfaction with graphics and content but a need for improvement regarding the navigability. An update of the platform was conducted in order to answer the needs expressed by participants.

Conclusion

We developed, with a participatory approach, a patient-centered transition program, which will be evaluated in a randomized controlled trial.

Toward Personalized Orthopedic Care: Validation of a Smart Knee Brace

Introduction

Wearable technology offers a promising solution to advance current rehabilitation strategies for post-operative orthopedic care. The aim of this study was to determine the level of agreement and concurrent validity of a smart knee brace compared to the gold standard measurement system GAITRite® for assessing lower limb gait parameters.

Methods

Thirty-four healthy participants were fitted with the smart knee brace (Digital Knee®) on their dominant limb. Gait parameters (stride length, stride time, and gait velocity) were measured simultaneously using the Digital Knee® and the GAITRite® electronic walkway. Two walks were performed at a comfortable speed and two at a fast-walking speed.

Results

At a comfortable walking speed, stride time was moderately valid (ICC2,1 = 0.66 s), and stride length and gait velocity demonstrated poor validity (ICC2,1 = 0.29; ICC2,1 = 0.41). All gait parameters demonstrated poor validity at a fast-walking speed (ICC2,1 = -0.16 to -0.01). Bias ranged from -0.08 to 0.28, with more clinically acceptable percentage errors at a comfortable walking speed (14.1-30%) versus at a fast-walking speed (26.4-42.6%). Gait velocity and stride length had substantially higher biases in the fast-walking speed compared to the comfortable walking speed (0.28 ± 0.39 m s-1 vs. 0.02 ± 0.21 m s-1; 0.15 ± 0.23 m vs. -0.04 ± 0.17 m). Limits of agreement were considered narrower for stride time compared to stride length and gait velocity.

Conclusion

The Digital Knee® is a promising approach to improving post-operative rehabilitation outcomes in patients with osteoarthritis. The Digital Knee® demonstrated good agreement and moderate concurrent validity for measuring gait metrics at a comfortable walking speed. These findings highlight the opportunity of the wearable sensor as an intervention for post-operative orthopedic care. This was a laboratory-based study; thus, further research is required to validate the wearable sensor in real-world contexts and in patients with knee pathologies. Further, refinement of the algorithm for measuring gait metrics at slow- and fast-walking speed with the Digital Knee® is warranted.

Effectiveness of self-reported management program of cancer patients

Objective

This study aimed to analyze the effect of Smart Cancer Care program on the quality of life and ease of chemotherapy continuation in cancer patients and the effect of additional tele-management on frequency of use and satisfaction with the Smart Cancer Care program.

Methods

'Smart Cancer Care' is a mobile program that allows cancer patients undergoing chemotherapy to report symptoms of adverse events and receive remote management. In this study, patients were randomly assigned to three groups: Group A, who received only classical face-to-face management; Group B, who used the Smart Cancer Care program as addition; and Group C, who used the Smart Cancer Care program and received telephone management. After 12 weeks of follow-up, the effectiveness of using the Smart Cancer Care program was analyzed by examining the quality of life, ease of maintaining chemotherapy, and unplanned hospital visits in each group. The frequency of use and satisfaction with the Smart Cancer Care program were also analyzed.

Results

Cancer patients who used the Smart Cancer Care program had 1.93-fold (1.15-3.25) higher overall quality of life than those who did not. This became 2.33-fold (1.34-4.04) higher when phone care was added. Patients with tele-management were significantly more likely to use the Smart Cancer Care program (odds ratio (OR) = 25.80; 95% confidence interval (CI), 11.28-58.97).

Conclusions

A mobile self-reported management program has a positive effect on the quality of life of cancer patients undergoing chemotherapy. Tele-management is conducive to active and effective use of this program.

A web-based resource for exercise training in children treated for brain tumours to improve cognitive sequelae: Development and usability

Improving cognitive sequelae in children treated for brain tumours (CTBT) requires accessible interventions. While instructor-led exercise in a hospital setting is efficacious, it is not extended to communities.

Objectives

We aimed to (i) develop a website with educational resources/tools for community health and fitness professionals (HFP) to deliver exercise for CTBT in community settings to improve cognition and (ii) assess its usability by community HFP. It was hypothesized that the website would be learnable, clear, satisfactory and efficient to deliver exercise.

Methods

A scoping review determined the state of eHealth resources to support exercise for CTBT and identified knowledge and resource gaps. Three focus groups with HFP who served cancer survivors in hospital or community settings (n = 13) identified user needs; content analysis identified themes. Gaps from the scoping review and themes from focus groups informed website content. A questionnaire assessed its usability by community HFP (n = 4). Descriptive statistics inferred the website's learnability, clarity, satisfaction and efficiency. Open-ended responses identified issues.

Results

The scoping review revealed a lack of eHealth resources supporting exercise to improve cognition in CTBT and education for HFP to deliver exercise. Six themes were identified in the focus groups. HFP rated the website as sufficiently learnable, clear, satisfactory and efficient. Two minor issues were reported and addressed.

Conclusion

The website marks one of the first eHealth resources to increase accessibility of intervention to improve cognitive sequelae and ultimately quality of life in CTBT. HFP also gain access to education and tools to deliver exercise in community settings.

Development of a Competency-Based Triage Education Application and Usability Testing for Triage Training Based on the Korean Triage and Acuity Scale

BACKGROUND

Improving the quality of emergency nursing care requires enhancing triage competency through education programs that integrate the content of the Korean Triage and Acuity Scale, a triage system specifically developed for the Korean context. Thus, this study developed a competency-based triage education application (CTEA), based on the Korean Triage and Acuity Scale, and evaluated its effectiveness through usability testing.

METHOD

The CTEA used various teaching methods and game mechanisms, including lectures, case studies, and video-based scenarios, to improve triage competency.

RESULTS

In the usability testing phase, the CTEA was refined through expert heuristic evaluation and user interviews conducted with the think-aloud method. Six themes were derived from the interviews, highlighting the need for a well-structured program with realistic scenarios, easy accessibility, gamification, functional improvements, and future triage educational applications.

CONCLUSION

These findings suggest that the CTEA is effective and satisfactory for users and can contribute to improving the triage competency of emergency nurses. [J Contin Educ Nurs. 2024;55(1):33-41.].

The German version of the mHealth App Usability Questionnaire (GER-MAUQ): Translation and validation study in patients with cardiovascular disease

Objective

In Germany, only a few standardized evaluation tools for assessing the usability of mobile Health apps exist so far. This study aimed to translate and validate the English patient version for standalone apps of the mHealth App Usability Questionnaire (MAUQ) into a German version.

Methods

Following scientific guidelines for translation and cross-cultural adaptation, the patient version for standalone apps was forward and back-translated from English into German by an expert panel. In total, 53 participants who were recruited as part of the beta testing process of the recently developed mHealth app HerzFit, answered the questions of the German version of the MAUQ (GER-MAUQ) and the System Usability Scale. Subsequently, a descriptive as well as a psychometric analysis was performed to test validity and reliability.

Results

After conducting three cognitive interviews, five items were modified. The values for Cronbach alpha for the entire questionnaire and the three subscales (0.966, 0.814, 0.910, and 0.909) indicate strong internal consistency. The correlation analysis revealed that the scores of the GER-MAUQ, the subscales and the SUS were strongly correlated with each other. The correlation coefficient of the SUS and the GER-MAUQ overall score was r = 0.854, P < 0.001 and the coefficients of the subscales and the SUS were r = 0.642, P < 0.001; r = 0.866, P < 0.001 and r = 0.643, P < 0.001.

Conclusions

We have developed a novel German version of the MAUQ and demonstrated it as a reliable and valid measurement tool for assessing the usability of standalone mHealth apps from the patients' perspective. The GER-MAUQ allows a new form of standardized assessment of usability of mHealth apps for patients with cardiovascular disease in Germany. Further research with a larger sample and other samples is recommended.

Development and Formative Evaluation of a Virtual Exercise Platform for a Community Fitness Center Serving Individuals With Physical Disabilities: Mixed Methods Study

BACKGROUND

People with disabilities experience numerous barriers to being physically active, such as transportation issues, a lack of trained exercise professionals who understand disabilities, and facility access. The use of a virtual exercise platform (VEP) may provide an alternative and limit certain barriers.

OBJECTIVE

The aim of this mixed method study was to evaluate user interaction (effectiveness, efficiency, and satisfaction), the strengths and weaknesses of the user interface, and the user experience with a VEP.

METHODS

Participants were recruited from a community fitness facility that offers programs for people with disabilities. Inclusion criteria were being older than 18 years, fluent in English, and availability of internet access. Features of the VEP included articles, prerecorded videos, live Zoom classes, web-based class registration, weekly progress tracking, incentives, and surveys. A one-on-one Zoom session was scheduled with each participant, during which they completed certain tasks: (1) create an account or login, (2) register for class, (3) join class, (4) add to calendar, and (5) complete surveys. As participants completed tasks, quantitative observations (time on task, task success, rate of task completion, and number of errors by users, which determined task difficulty), qualitative observations were made and interviews were conducted at the end of the session. The "concurrent think-aloud" method was encouraged by the moderator to gauge participants' thoughts as they worked through testing. Participants also completed the System Usability Scale (SUS) and Questionnaire for User Interface Satisfaction (QUIS).

RESULTS

A total of 5 people with disabilities (3 male, 2 female), aged 36-78 (mean 54) years, with education levels from high school to PhD, were recruited. Devices used for testing included a laptop (n=3), a Chromebook (n=1), and a desktop (n=1). All participants completed tasks #1 and #2 without errors but could not complete task #4. One participant completed task #5 with difficulty and another completed task #3 with difficulty. The average time to complete each task was: (1) 82 seconds (55-110), (2) 11 seconds (4-21), (3) 9 seconds (5-27), and (4) 921.5 seconds (840-958). The mean SUS score was 86.5/100, and the mean user QUIS score was 8.08 out of 10. Qualitative observations indicated that the system was simple, user-friendly, and accessible.

CONCLUSIONS

People with disabilities reported high usability and user satisfaction with the web-based exercise platform, and the system appears to be an efficient and effective physical activity option.

An Online Psychological Program for Adolescents and Young Adults With Headaches: Iterative Design and Rapid Usability Testing

BACKGROUND

Headache disorders are common, debilitating health problems. Cognitive-behavioral therapy (CBT) is recommended but rarely easily available. With the use of the internet and communication technologies among youth and young adults, these individuals could be self-trained in CBT skills. There is an increasing number of internet-based interventions for headaches, but there has been little research into the usability of these interventions because evaluating usability across the intervention development life cycle is costly. We developed an internet-based CBT program, the Specialized Program for Headache Reduction (SPHERE). While developing it, we aimed to improve SPHERE through rapid usability testing cycles.

OBJECTIVE

This study aims to presents a rapid and affordable usability testing approach that can be performed throughout the intervention development life cycle. This paper also provides evidence of the usability of SPHERE.

METHODS

We used the "think aloud" usability testing method based on Krug's approach to test user interaction within a lab setting. This was followed by a short posttest interview. We planned to test SPHERE with 3-5 participants testing the same part of the program each cycle. Both the design and development team and the research team actively participated in the usability testing process. Observers independently identified the top 3 usability issues, rated their severity, and conducted debriefing sessions to come to consensus on major issues and generate potential solutions.

RESULTS

The testing process allowed major usability issues to be identified and rectified rapidly before piloting SPHERE in a real-world context. A total of 2 cycles of testing were conducted. Of the usability issues encountered in cycles 1 and 2, a total of 68% (17/25) and 32% (12/38), respectively, were rated as major, discussed, and fixed.

CONCLUSIONS

This study shows that rapid usability testing is an essential part of the design process that improves program functionality and can be easy and inexpensive to undertake.

Exploring usability characteristics in computer-based digital health technologies for family caregivers of people with chronic progressive conditions: a scoping review protocol

OBJECTIVE

The objective of this scoping review is to map the literature on how usability is considered during the design and/or evaluation of computer-based digital health technologies for family caregivers of persons with chronic progressive conditions.

INTRODUCTION

Computer-based digital health technologies offer convenient alternatives for delivering interventions to caregivers of people with chronic progressive conditions. Usability is a critical component of good practice in developing and implementing health and social care technologies; however, we need to determine whether usability is incorporated in the design and/or evaluation of computer-based digital health technologies for caregivers of people with chronic progressive conditions. Within this context, a broad overview of the existing literature on usability in computer-based digital health technologies is needed.

INCLUSION CRITERIA

We will include studies published from 2012 to the present that describe usability characteristics of computer-based digital health technologies targeting adult (≥18 years old) family caregivers of people with chronic progressive conditions, regardless of study design or setting.

METHODS

We will use the JBI methodology for scoping reviews. We will conduct searches of MEDLINE (Ovid), PsycINFO (Ovid), CINAHL (EBSCOhost), and Web of Science Core Collection to capture eligible studies. After the results are deduplicated, 2 independent reviewers will assess each study for eligibility and extract data from the included studies. Conflicts will be resolved through discussion or with a third reviewer. Data analysis will use a textual narrative synthesis approach.

REVIEW REGISTRATION

Open Science Framework osf.io/w4vk5.

Usability and feasibility of E-nergEYEze: a blended vision-specific E-health based cognitive behavioral therapy and self-management intervention to reduce fatigue in adults with visual impairment

BACKGROUND

Over 50% of adults with visual impairment experience severe fatigue. Therefore, we developed a guided E-health intervention based on cognitive behavioral therapy and self-management to reduce fatigue in this population. This pilot study evaluated the usability, feasibility, fidelity and potential effectiveness of E-nergEYEze.

METHODS

E-nergEYEze was developed by a design team and customized by conducting a pilot study using an iterative development strategy. The intervention was first tested in a usability study among adults with visual impairment (n = 5). Participants were asked to think-aloud while exploring the intervention features and a semi-structured interview was performed afterwards. Subsequently, the enhanced intervention was tested in a feasibility study. Adults with visual impairment and severe fatigue (n = 10) followed the intervention partially with guidance from a social worker and one-time computer trainer support. Fatigue severity (Checklist Individual Strength), fatigue impact (Modified Fatigue Impact Scale) and cognitive behavioral therapy skills (Competencies of Cognitive Therapy Scale-Self Report) were measured at baseline and at three months follow-up and analyzed with the Wilcoxon signed-rank test. The intervention was evaluated through evaluation forms.

RESULTS

The usability study resulted in adjustments to content and lay-out with regard to optically shortened text sentences, separate pages for information and assignments with one read-aloud audio and an additional descriptive explanation of page content. Digital challenges were overcome with mandatory computer training and e-platform modifications. The feasibility study showed a positive trend in reducing fatigue severity (Z -6.108; P < .001; SD 8.4), impact of fatigue (Z - 4.451; P < .001; SD 11.4) and cognitive behavioral therapy skills (Z -2.278; P = .023; SD 19.3). Participants gave useful feedback regarding accessibility, content and guidance, with an overall positive experience. The intervention was rated with a median score of 8 (range 7-10).

CONCLUSION

We developed, evaluated and optimized E-nergEYEze by applying a user-centered and iterative approach. E-nergEYEze showed a promising trend to reduce fatigue severity and impact of fatigue and to increase cognitive behavioral therapy skills. The study methods were feasible and the fidelity of the intervention protocol was suitable. Performing a randomized controlled trial is warranted to give insight into whether E-nergEYEze is cost-effective in reducing severe fatigue in adults with visual impairment.

TRIAL REGISTRATION

International Clinical Trial Registry Platform: NL7764. Date registered: 28-05-2019.

Feasibility and Usability of the Job Adjustment Mobile App for Pregnant Women: Longitudinal Observational Study

BACKGROUND

Working pregnant women often need to adjust their physically demanding jobs for a healthy pregnancy. However, uncertainty about the extent of these adjustments can hinder their effectiveness. To address this, we developed the Job Adjustment mobile app, which allows users to input job and health details to generate a variety of personalized action plans. As this is the first version of the app, assessing its feasibility and usability is crucial.

OBJECTIVE

This study aims to verify the feasibility and usability of the Job Adjustment mobile app.

METHODS

A longitudinal observational study was conducted on pregnant Japanese women who were allowed to use the app anytime from 12 to 34 weeks of gestation; they received reminder emails every 2 weeks encouraging app use. A questionnaire was administered before app use and at 20 and 32 weeks of gestation. Feasibility was evaluated across 4 domains: implementation, demand, acceptability, and adverse events. Implementation was evaluated based on 3 parameters: dropout rate, initial reminder email receipt rate, and adherence rate (measured as pregnant women who used the app at intervals of 2.5 weeks or less). Demand was measured by intervals between use and intervals between log-in, and participants answered 15 questions to assess acceptability. Adverse events were assessed by analyzing the degree of anxiety related to work. Demographic data were analyzed to determine any statistically significant differences in intervals between uses. Usability was evaluated using the System Usability Scale.

RESULTS

The analysis included 66 pregnant women, and 61% (n=40) of them were multipara. The dropout rate, adherence rate, and initial reminder email receipt rate were 18% (13/71), 44% (29/66), and 79% (52/66) respectively. The median intervals between use and intervals between log-in were 2.94 (IQR 2.00-5.13) weeks and 2.28 (IQR 1.81-4.00) weeks, respectively. Overall, 60% (35/58) to 90% (52/58) of the participants responded positively to all 15 questions assessing acceptability, and no anxiety regarding work was recorded. The mean System Usability Scale score was 66.1 points. Multipara women had significantly longer intervals between app use compared to primipara women (P=.01).

CONCLUSIONS

The results demonstrated acceptable levels of feasibility and usability of the app. However, the low adherence rates, especially among multipara women, suggest the need for modifications to reduce the time burden of the app. Further research should explore more effective and acceptable intervals between use and timing, involving a larger sample and accounting for diverse characteristics of pregnant women.

TRIAL REGISTRATION

UMIN Clinical Trials Registry UMIN000042943; https://tinyurl.com/ydrchfas.

Usability and Overall Perception of a Health Bot for Nutrition-Related Questions for Patients Receiving Bariatric Care: Mixed Methods Study

BACKGROUND

Currently, over 4000 bariatric procedures are performed annually in Switzerland. To improve outcomes, patients need to have good knowledge regarding postoperative nutrition. To potentially provide them with knowledge between dietetic consultations, a health bot (HB) was created. The HB can answer bariatric nutrition questions in writing based on artificial intelligence.

OBJECTIVE

This study aims to evaluate the usability and perception of the HB among patients receiving bariatric care.

METHODS

Patients before or after bariatric surgery tested the HB. A mixed methods approach was used, which consisted of a questionnaire and qualitative interviews before and after testing the HB. The dimensions usability of, usefulness of, satisfaction with, and ease of use of the HB, among others, were measured. Data were analyzed using R Studio (R Studio Inc) and Excel (Microsoft Corp). The interviews were transcribed and a summary inductive content analysis was performed.

RESULTS

A total of 12 patients (female: n=8, 67%; male: n=4, 33%) were included. The results showed excellent usability with a mean usability score of 87 (SD 12.5; range 57.5-100) out of 100. Other dimensions of acceptability included usefulness (mean 5.28, SD 2.02 out of 7), satisfaction (mean 5.75, SD 1.68 out of 7), and learnability (mean 6.26, SD 1.5 out of 7). The concept of the HB and availability of reliable nutrition information were perceived as desirable (mean 5.5, SD 1.64 out of 7). Weaknesses were identified in the response accuracy, limited knowledge, and design of the HB.

CONCLUSIONS

The HB's ease of use and usability were evaluated to be positive; response accuracy, topic selection, and design should be optimized in a next step. The perceptions of nutrition professionals and the impact on patient care and the nutrition knowledge of participants need to be examined in further studies.

Gamification to Improve Medication Adherence: A Mixed-method Usability Study for MedScrab

Medication non-adherence is a prevalent healthcare problem with poor health outcomes and added healthcare costs. MedScrab, a gamification-based mHealth app, is the first attempt to deliver crucial life-saving medication information to patients and increase their medication adherence. The paper presents the development of MedScrab and a two-phase mixed-method usability evaluation of MedScrab. Phase I qualitatively evaluated MedScrab using a think-aloud protocol for its usability. With 51 participants, qualitative data analysis of Phase I revealed two themes: positive functionality of the app and four areas of improvement. The improvement recommendations were incorporated into MedScrab's design. Phase I also validated a widely used mHealth App Usability Questionnaire (MAUQ). Quantitative data analysis of Phase I reduced the original 18-item MAUQ scale to a 15-item scale with two factors: ease of use (4 items) and usefulness and satisfaction (11 items). Phase II surveyed 83 participants from Amazon's Mechanical Turk using a modified MAUQ. The modified MAUQ scale showed strong internal consistency (Cronbach alpha = 0.959) and high factor loadings (between 0.623 and 0.987). The study design of the usability evaluation can serve as a methodological guide for designing, evaluating, and improving mHealth apps.The usability study showed that MedScrab was perceived as ease of use (6.24 out of 7) with high usefulness and satisfaction (5.72 out of 7). The quantitative data analysis results support the use of the modified MAUQ as a valid instrument to measure the usability of the MedScrab. However, the instrument should be used with adaptation based on the app's characteristics.

Avazum app usability testing

PURPOSE

To verify the effectiveness, efficiency, and satisfaction in the usability test of the tinnitus assessment application.

METHODS

This is a descriptive usability test study that assessed the satisfaction, effectiveness, and efficiency of the application. The test was carried out virtually via Google Forms. First, the participants received instructions on how to access and use Avazum, through texts and illustrative images. Afterward, the users used the application, observing its functions and usability. Next, the participants answered the usability questionnaire. Statistical analysis was performed using the Statistical Package for the Social Sciences, version 21.0, to perform descriptive analysis such as frequency, percentages of data in the System Usability Questionnaire (SUS), and analysis in the Net Promoter Score (NPS).

RESULTS

Thus, the overall mean SUS score was consistent with satisfactory usability, which implies that the application does not present serious usability problems. Also, the overall NPS percentage indicated very good user satisfaction, with a good percentage of promoting users. As far as effectiveness is concerned, it was found that Avazum reached its goals, besides being efficient, as it uses clear language and is comfortably used.

CONCLUSION

Avazum obtained very good satisfaction from users, in addition to achieving its goals. It proved to be effective, has clear language, and is comfortably used, adducing efficiency in the multidisciplinary evaluation of tinnitus.

HD-DRUM, a Tablet-Based Drumming Training App Intervention for People With Huntington Disease: App Development Study

BACKGROUND

Huntington disease (HD) is a neurodegenerative condition that leads to progressive loss of cognitive-executive and motor functions, largely due to basal ganglia (BG) atrophy. Currently, there are no therapeutic interventions tailored to address executive and motor dysfunction in people with HD. Music-based interventions may aid executive abilities by compensating for impaired BG-reliant timing and rhythm generation using external rhythmic beats. Here, we applied an integrated knowledge translation (IKT) framework to co-design a tablet-based rhythmic drumming training app (HD-DRUM) to stimulate executive and motor abilities in people with HD.

OBJECTIVE

The primary aim was to develop the HD-DRUM app for at-home use that addressed the accessibility needs of people with HD and allowed for the quantification of performance improvements and adherence for controlled clinical evaluation.

METHODS

The IKT framework was applied to iteratively refine the design of HD-DRUM. This process involved 3 phases of knowledge user engagement and co-design: a web-based survey of people with HD (n=29) to inform about their accessibility needs, usability testing of tablet-based touch screens as hardware solutions, and usability testing of the design and build of HD-DRUM to meet the identified accessibility needs of people affected by HD and their clinicians (n=12).

RESULTS

The survey identified accessibility problems due to cognitive and motor control impairments such as difficulties in finding and navigating through information and using PC keyboards and mouses to interact with apps. Tablet-based touch screens were identified as feasible and accessible solutions for app delivery. Key elements to ensure that the app design and build met the needs of people with HD were identified and implemented. These included the facilitation of intuitive navigation through the app using large and visually distinctive buttons; the use of audio and visual cues as training guides; and gamification, positive feedback, and drumming to background music as a means to increase motivation and engagement. The co-design development process resulted in the proof-of-concept HD-DRUM app that is described here according to the Template for Intervention Description and Replication checklist. HD-DRUM can be used at home, allowing the quantification of performance improvements and adherence for clinical evaluation, matching of training difficulty to users' performance levels using gamification, and future scale-up to reach a wide range of interested users.

CONCLUSIONS

Applying an IKT-based co-design framework involving knowledge user engagement allowed for the iterative refinement of the design and build of the tablet-based HD-DRUM app intervention, with the aim of stimulating BG-reliant cognitive and motor functions. Mapping the intervention against the Template for Intervention Description and Replication framework to describe complex interventions allowed for the detailed description of the HD-DRUM intervention and identification of areas that required refinement before finalizing the intervention protocol.

A Web-Based Tool to Report Adverse Drug Reactions by Community Pharmacists in Australia: Usability Testing Study

BACKGROUND

Adverse drug reactions (ADRs) are unintended and harmful events associated with medication use. Despite their significance in postmarketing surveillance, quality improvement, and drug safety research, ADRs are vastly underreported. Enhanced digital-based communication of ADR information to regulators and among care providers could significantly improve patient safety.

OBJECTIVE

This paper presents a usability evaluation of the commercially available GuildCare Adverse Event Recording system, a web-based ADR reporting system widely used by community pharmacists (CPs) in Australia.

METHODS

We developed a structured interview protocol encompassing remote observation, think-aloud moderating techniques, and retrospective questioning to gauge the overall user experience, complemented by the System Usability Scale (SUS) assessment. Thematic analysis was used to analyze field notes from the interviews.

RESULTS

A total of 7 CPs participated in the study, who perceived the system to have above-average usability (SUS score of 68.57). Nonetheless, the structured approach to usability testing unveiled specific functional and user interpretation issues, such as unnecessary information, lack of system clarity, and redundant data fields-critical insights not captured by the SUS results. Design elements like drop-down menus, free-text entry, checkboxes, and prefilled or auto-populated data fields were perceived as useful for enhancing system navigation and facilitating ADR reporting.

CONCLUSIONS

The user-centric design of technology solutions, like the one discussed herein, is crucial to meeting CPs' information needs and ensuring effective ADR reporting. Developers should adopt a structured approach to usability testing during the developmental phase to address identified issues comprehensively. Such a methodological approach may promote the adoption of ADR reporting systems by CPs and ultimately enhance patient safety.

User-Centered Development of a Mobile App to Assess the Quality of Life of Patients With Cancer: Iterative Investigation and Usability Testing

BACKGROUND

The treatment for cancer can have a negative impact not only on physical well-being but also on mental health and the quality of life (QoL). Health apps enable the monitoring of different parameters, but to date, there are only few that support patients with cancer and none that focuses on the assessment of QoL. Furthermore, patients as stakeholders are often only integrated at the late stage of the development process, if at all.

OBJECTIVE

The aim of this research was to develop and evaluate a smartphone app (Lion-App) to enable patients with cancer to autonomously measure the QoL with an iterative, user-centered approach.

METHODS

Patients with cancer were involved in a 3-stage process from conceptualization to the point when the app was available on the tester's private device. First, focus groups with members (N=21) of cancer support groups were conducted to understand their expectations and needs. Thereafter, individual tests were performed. After developing a prototype that incorporated findings from the focus groups, a second test cycle was conducted, followed by a beta test lasting 2 months. In our app, the QoL can be assessed via a patient diary and an integrated questionnaire. Through all stages, usability was evaluated using the modular extended version of the User Experience Questionnaire (UEQ+), including the calculation of a key performance indicator (KPI). If possible, the impact of sex on the results was evaluated. As part of the beta test, usage rates as well as age-dependent differences were also assessed.

RESULTS

A total of 21 participants took part in the initial 3 focus groups. In the subsequent usability testing (N=18), 17 (94%) participants rated their impression through the UEQ+, with a mean KPI of 2.12 (SD 0.64, range: -3 to 3). In the second usability test (N=14), the mean KPI increased to 2.28 (SD=0.49). In the beta test, the usage rate of 19 participants was evaluated, of whom 14 (74%) also answered the UEQ+ (mean KPI 1.78, SD 0.84). An influence of age on the number of questionnaire responses in Lion-App was observed, with a decrease in responses with increasing age (P=.02). Sex-dependent analyses were only possible for the first usability test and the beta test. The main adjustments based on user feedback were a restructuring of the diary as well as integration of a shorter questionnaire to assess the QoL.

CONCLUSIONS

The iterative, user-centered approach for development and usability testing resulted in positive evaluations of Lion-App. Our app was rated as suitable for everyday use to monitor the QoL of patients with cancer. Initial results indicated that the sex and age of participants seem to play only a minor role.

Usability evaluation of the Agente Escuta application: translational research

PURPOSE

To evaluate the usability and satisfaction of users with the interface of the 'Agente Escuta' application, in addition to identifying problems and possibilities for improvement.

METHODS

Descriptive exploratory translational study, characterized by a usability test with a quantitative and qualitative approach, subdivided into three stages: (I) prior evaluation of usability by 10 judges, including students, primary care professionals, professors and researchers in Information Technology and Speech Therapy; (II) evaluation of the application by the target audience, that is, community health agents from six municipalities in Rio Grande do Norte; (III) evaluation of the satisfaction of the agents who used the application in their work routine. The System Usability Scale and the Net Promoter Score were used, in addition to a qualitative evaluation of the opinions.

RESULTS

Usability was rated as excellent by judges, regardless of category. In the evaluation by community health agents, usability was considered good and there was no effect of the city of origin. It was found that the perception of the judges and the target audience were different, with a lower score for the participants in the second stage. However, most would give positive publicity to the product. The heuristic with the highest score was ease of memorization and participants in the third stage were interested in continuing to use the tool in practice, even after the end of the study.

CONCLUSION

The Agente Escuta prototype showed good usability and satisfaction and aspects that could be improved in future solutions were identified.

The "Breast-4Y" web app for breast cancer prevention at young age: Development, evaluation, and validation

BACKGROUND

Improving knowledge of risk factors, signs and symptoms positively influence an individual's intention to acquire healthy lifestyle behaviors to prevent breast cancer, to identify breast cancer risk early and seek health advice early. The aim of the present study was to develop and to assess the usability and quality of a web-app (Breast-4Y) designed specifically for younger women to improve their knowledge about breast cancer risk factors and symptoms as well as protective behaviors to reduce their risk.

METHODS

The development of the Breast-4Y web-app, based on Health Belief Model and the Behaviour Change Wheel, comprised four steps: i) analysis of scientific literature regarding breast cancer risk factors, symptoms and preventive lifestyle behaviors; ii) design of web-app; iii) content review by experts; iv) quality and usability assessment by the end-users (n = 20). Web-app usability was evaluated using the Spanish version of the System Usability Scale for the Assessment of Electronic Tools and the quality using the Spanish version of uMARS.

RESULTS

The contents were rated 4.25 or above, indicating high content quality. The mean usability score was 84.9 (SD 12.4), the mean rating for objective quality and subjective quality dimensions mean were 4.2 (SD 0.4) and 3.4 (0.777) respectively, indicating excellent usability and quality. Positive associations (p < .05) were observed between numbers of app used and attitudes to change (r = 0.479), and intention to change (r = 0.539). Strong positive associations (p < .001) were also observed between usability satisfaction and attitudes to change (r = 0.584), intention to change (r = 0.656), and help-seeking (r = 0.656).

CONCLUSIONS

This study provided evidence that Breast-4Y has adequate contents, high quality, and usability. Breast-4Y can be tested in pragmatic trials to assess their effectiveness to reduce the risk of breast cancer, raise women's awareness of breast cancer risk factors, increase women's knowledge of breast cancer symptoms, and adopt protective lifestyle behaviors.

Development and usability of a Mobile Interactive Application (VCPW) for Vascular Crisis Prewarning after Skin Flap Transplantation

Background

In microsurgical tissue transfer, skin flap transplantation is frequently used to heal the surface of a wound. Effective microcirculation surveillance of the skin flap is crucial. However, with traditional monitoring methods-that is, clinical observation-vascular crisis can still occur, thereby impairing postoperative recovery. A smartphone application is required to assist health care professionals in the standardized collection of flap perfusion parameters for flap management.

Methods

The Vascular Crisis Prewarning Application was created using a design science research methodology that prioritizes users and problems. The system usability scale was used to assess the application's usability among medical practitioners. The application was used at the clinic from December 2020 to September 2022. The unplanned return to the operating room, time to diagnose vascular crisis, and flap survival rate were compared with and without the application.

Results

The application consisted of 5 modules: patient addition and basic information entry, flap labeling, flap observation, crisis warning, and case archiving. The average rating for the application's usability among medical practitioners was 97.95 score (SD 2.36). With the application, the time to detect vascular crisis reduced from 26.71 to 16.26 h (P < 0.001), the unplanned return to the operation room increased from 8.18% to 10.24% (P = 0.587), and the flap survival rate went from 94.55% to 99.21% (P = 0.083).

Conclusions

An easy-to-use flap perfusion monitoring and prewarning application for medical practitioners was produced using a user-centered development method. The application provided a more standardized and accurate platform for data collection in flap management and reduced the time to detect vascular crisis. Larger cohort studies are required in the future to better assess the full potential of the application.

Diagnostic Performance, Triage Safety, and Usability of a Clinical Decision Support System Within a University Hospital Emergency Department: Algorithm Performance and Usability Study

Background

Computerized clinical decision support systems (CDSSs) are increasingly adopted in health care to optimize resources and streamline patient flow. However, they often lack scientific validation against standard medical care.

Objective

The purpose of this study was to assess the performance, safety, and usability of a CDSS in a university hospital emergency department setting in Kuopio, Finland.

Methods

Patients entering the emergency department were asked to voluntarily participate in this study. Patients aged 17 years or younger, patients with cognitive impairments, and patients who entered the unit in an ambulance or with the need for immediate care were excluded. Patients completed the CDSS web-based form and usability questionnaire when waiting for the triage nurse's evaluation. The CDSS data were anonymized and did not affect the patients' usual evaluation or treatment. Retrospectively, 2 medical doctors evaluated the urgency of each patient's condition by using the triage nurse's information, and urgent and nonurgent groups were created. The International Statistical Classification of Diseases, Tenth Revision diagnoses were collected from the electronic health records. Usability was assessed by using a positive version of the System Usability Scale questionnaire.

Results

In total, our analyses included 248 patients. Regarding urgency, the mean sensitivities were 85% and 19%, respectively, for urgent and nonurgent cases when assessing the performance of CDSS evaluations in comparison to that of physicians. The mean sensitivities were 85% and 35%, respectively, when comparing the evaluations between the two physicians. Our CDSS did not miss any cases that were evaluated to be emergencies by physicians; thus, all emergency cases evaluated by physicians were evaluated as either urgent cases or emergency cases by the CDSS. In differential diagnosis, the CDSS had an exact match accuracy of 45.5% (97/213). The usability was good, with a mean System Usability Scale score of 78.2 (SD 16.8).

Conclusions

In a university hospital emergency department setting with a large real-world population, our CDSS was found to be equally as sensitive in urgent patient cases as physicians and was found to have an acceptable differential diagnosis accuracy, with good usability. These results suggest that this CDSS can be safely assessed further in a real-world setting. A CDSS could accelerate triage by providing patient-provided data in advance of patients' initial consultations and categorize patient cases as urgent and nonurgent cases upon patients' arrival to the emergency department.

Clinician Perceptions of Transition From Legacy Electronic Health Record to MHS GENESIS: A Pilot Study

INTRODUCTION

The implementation of a new electronic health record (EHR) presents significant challenges as users navigate a new interface. Our institution was an early adopter of MHS GENESIS (MHSG), the MHS's new EHR. This study investigated provider perceptions of usability and the prevalence of burnout during an EHR transition from the Legacy system (LEHR) utilizing online, anonymous surveys before implementation and at several points post-implementation.

MATERIALS AND METHODS

Clinician satisfaction, perceptions of EHR impact on safety, communication, reliability, and chart completion were assessed on five-point Likert scales. Usability was assessed using the validated System Usability Scale (SUS). Burnout prevalence was assessed using a validated single-item measure. Data were gathered via online, anonymous surveys before implementation and at 1, 3-6, and 9-12 months post "Go-Live."

RESULTS

Of 367 clinicians, 56 responded to the baseline survey; on average, 29% of responses were positive (i.e., satisfied or very satisfied) with LEHR. Following implementation, an average of 47%, 37%, and 47% of responses were positive for MHSG at 1 months (n = 42), 6 months (n = 55), and 12 months (n = 30), respectively. The mean SUS for LEHR was 48 (on a scale of 1-100); for MHSG, the mean SUS was 48 (1 month), 41 (6 months), and 44 (12 months). The burnout rate was reported as 39% (baseline), 26% (1 month), 33% (6 months), and 37% (12 months).

CONCLUSIONS

Perceptions of system usability, physician satisfaction, and burnout were not remarkably different between the two systems. Study results imply a need for further investigation as to why most clinicians continue to be unsatisfied with MHSG and wish to return to LEHR and whether or how this EHR transition contributed to burnout or if other factors are more contributory in this population. Future studies could also explore whether changes in the clinician interface, training, or implementation process impact clinician satisfaction, burnout, and desire to return to LEHR, or if these measures change with longer follow-up or in more varied populations.

The feasibility of using a digital tool to enhance mobilisation following abdominal cancer surgery-a non-randomised controlled trial

BACKGROUND

Early mobilisation is strongly recommended following abdominal cancer surgery, but evidence on how to structure early mobilisation to improve outcomes and support patient adherence is lacking. Pedatim® (Phystec) is a novel digital tool designed to support mobilisation in hospital settings using prescribed activities and goals on a tablet. The aim of this study was to evaluate the feasibility of the Pedatim tablet to enhance mobilisation following abdominal cancer surgery.

METHODS

In a non-randomised feasibility trial design, participants were recruited between January and May 2022 at Karolinska University Hospital, Sweden. Participants used a Pedatim tablet from postoperative day 1 (POD 1) until hospital discharge. The primary objective was to evaluate process feasibility, regarding recruitment, compliance, and acceptability. Recruitment was measured by percentage of available patients included, eligibility criteria sufficiency, and number of dropouts. Compliance was measured by number of patients using versus not using the board. Acceptability was measured using the System Usability Scale. The secondary objective was to evaluate scientific feasibility, defined as an indication of treatment effects where physical activity was assessed using an activPAL accelerometer. Unforeseen events relating to the tablet were also registered.

RESULTS

Based on predetermined feasibility criteria, the overall study design was determined to be feasible regarding recruitment as 69% accepted participation (n = 20), compliance was 95%, and the acceptability mean score was high (77/100). Eligibility criteria were not feasible as 79% (n = 108) of available patients were excluded. The intervention was determined to be scientifically feasible, mean steps per day increased from 623 (SD 766) to 1823 (SD 1446), and mean sit-to-stand transitions per day increased from 11 (SD 8) to 29 (SD 12) POD 1-4. Technical issues emerged, highlighting the need for available technical support and "user champions" among healthcare professionals on the ward.

CONCLUSIONS

Using the Pedatim tablet to enhance mobilisation following abdominal cancer surgery was deemed feasible, but a randomised controlled trial is needed to determine the tool's effectiveness. The study process was determined to be feasible with revisions of the eligibility criteria needed before a future trial. Involving healthcare professionals and providing available technical support are important for future implementation.

A Tablet-Based App to Support Nursing Home Staff in Delivering an Individualized Cognitive and Physical Exercise Program for Individuals With Dementia: Mixed Methods Usability Study

BACKGROUND

The promotion of physical activity in individuals with dementia living in nursing homes is crucial for preserving physical and cognitive functions and the associated quality of life. Nevertheless, the implementation of physical activity programs in this setting is challenging, as the time and expertise of nursing home staff are limited. This situation was further exacerbated by the COVID-19 pandemic. Mobile health apps may be a sustainable approach to overcome these challenges in the long term. Therefore, the Individualized Cognitive and Physical Exercise-App (the InCoPE-App) was developed to support nursing home staff in delivering and implementing tailored cognitive and physical exercise training for individuals with dementia.

OBJECTIVE

This study aims to assess the usability of the InCoPE-App in terms of user performance and user perception in a laboratory setting using a mixed methods approach.

METHODS

Nursing home staff were encouraged to perform 5 basic tasks within the InCoPE-App. Their thoughts while using the app were captured by implementing a think aloud protocol. Then, participants completed the System Usability Scale questionnaire. The think aloud transcripts were qualitatively evaluated to unveil usability issues. All identified issues were rated in terms of their necessity to be fixed. Task completion (ie, success rate and time) and perceived usability were evaluated descriptively.

RESULTS

A total of 14 nursing home employees (mean age 53.7, SD 10.6 years; n=13, 93% women) participated in the study. The perceived usability of the InCoPE-App, as assessed by the System Usability Scale questionnaire, can be rated as "good." The main usability issues concerned navigation logic and comprehensibility of app content.

CONCLUSIONS

The InCoPE-App is a user-friendly app that enables nursing home staff to deliver and implement cognitive and physical exercise training for individuals with dementia in nursing homes. The InCoPE-App can be used with little training, even by people aged ≥50 years, who may have low digital literacy. To achieve sustainable use and high user satisfaction of the InCoPE-App in the long term, it should be implemented and evaluated in a field study.

The Constrained-Disorder Principle Assists in Overcoming Significant Challenges in Digital Health: Moving from "Nice to Have" to Mandatory Systems

The success of artificial intelligence depends on whether it can penetrate the boundaries of evidence-based medicine, the lack of policies, and the resistance of medical professionals to its use. The failure of digital health to meet expectations requires rethinking some of the challenges faced. We discuss some of the most significant challenges faced by patients, physicians, payers, pharmaceutical companies, and health systems in the digital world. The goal of healthcare systems is to improve outcomes. Assisting in diagnosing, collecting data, and simplifying processes is a "nice to have" tool, but it is not essential. Many of these systems have yet to be shown to improve outcomes. Current outcome-based expectations and economic constraints make "nice to have," "assists," and "ease processes" insufficient. Complex biological systems are defined by their inherent disorder, bounded by dynamic boundaries, as described by the constrained disorder principle (CDP). It provides a platform for correcting systems' malfunctions by regulating their degree of variability. A CDP-based second-generation artificial intelligence system provides solutions to some challenges digital health faces. Therapeutic interventions are held to improve outcomes with these systems. In addition to improving clinically meaningful endpoints, CDP-based second-generation algorithms ensure patient and physician engagement and reduce the health system's costs.

Methodologies for Evaluating the Usability of Rehabilitation Technologies Aimed at Supporting Shared Decision-Making: Scoping Review

BACKGROUND

The field of rehabilitation has seen a recent rise in technologies to support shared decision-making (SDM). Usability testing during the design process of SDM technologies is needed to optimize adoption and realize potential benefits. There is variability in how usability is defined and measured. Given the complexity of usability, a thorough examination of the methodologies used to measure usability to develop the SDM technologies used in rehabilitation care is needed.

OBJECTIVE

This scoping review aims to answer the following research questions: which methods and measures have been used to produce knowledge about the usability of rehabilitation technologies aimed at supporting SDM at the different phases of development and implementation? Which parameters of usability have been measured and reported?

METHODS

This review followed the Arksey and O'Malley framework. An electronic search was performed in the Ovid MEDLINE, Embase, CINAHL, and PsycINFO databases from January 2005 up to November 2020. In total, 2 independent reviewers screened all retrieved titles, abstracts, and full texts according to the inclusion criteria and extracted the data. The International Organization for Standardization framework was used to define the scope of usability (effectiveness, efficiency, and satisfaction). The characteristics of the studies were outlined in a descriptive summary. Findings were categorized based on usability parameters, technology interventions, and measures of usability.

RESULTS

A total of 38 articles were included. The most common SDM technologies were web-based aids (15/33, 46%). The usability of SDM technologies was assessed during development, preimplementation, or implementation, using 14 different methods. The most frequent methods were questionnaires (24/38, 63%) and semistructured interviews (16/38, 42%). Satisfaction (27/38, 71%) was the most common usability parameter mapped to types of SDM technologies and usability evaluation methods. User-centered design (9/15, 60%) was the most frequently used technology design framework.

CONCLUSIONS

The results from this scoping review highlight the importance and the complexity of usability evaluation. Although various methods and measures were shown to be used to evaluate the usability of technologies to support SDM in rehabilitation, very few evaluations used in the included studies were found to adequately span the selected usability domains. This review identified gaps in usability evaluation, as most studies (24/38, 63%) relied solely on questionnaires rather than multiple methods, and most questionnaires simply focused on the usability parameter of satisfaction. The consideration of end users (such as patients and clinicians) is of particular importance for the development of technologies to support SDM, as the process of SDM itself aims to improve patient-centered care and integrate both patient and clinician voices into their rehabilitation care.

Design and development of an eHealth intervention to support self-management in people with musculoskeletal disorders - 'eHealth: It's TIME': a study protocol

Background: Musculoskeletal disorders (MSDs) are a leading cause of global morbidity, with the burden expected to increase in the near future. Self-management, with the support of healthcare professionals, is recommended for many MSDs. However, frequent clinical contact is not feasible. Previous research has highlighted the need for a co-designed eHealth-mediated self-management follow-up support intervention which integrates remote monitoring and behavioural change. Thus, the current study aims to develop and design a user-centred, eHealth-mediated self-management support prototype for people with MSDs. Methods: A three-step, iterative system development cycle will be utilised to develop and design the "eHealth: It's TIME prototype". The three-step process will include creating website features and content using two sequential focus groups with people with MSDs (n = 6 - 8); heuristic testing using the 10 heuristic principles of Nielsen (n = 5); and usability testing through in-person 60-minute interviews with people with MSDs (n = 3 - 5) and musculoskeletal physiotherapists (n = 3 - 5). Conclusion: The eHealth: It's TIME prototype will be a systematically developed, follow-up self-management support intervention guided by behavioural change theory and the preferences of end users.

A Novel Mobile App for Personalized Dietary Advice Leveraging Persuasive Technology, Computer Vision, and Cloud Computing: Development and Usability Study

BACKGROUND

The Australian Dietary Guidelines (ADG) translate the best available evidence in nutrition into food choice recommendations. However, adherence to the ADG is poor in Australia. Given that following a healthy diet can be a potentially cost-effective strategy for lowering the risk of chronic diseases, there is an urgent need to develop novel technologies for individuals to improve their adherence to the ADG.

OBJECTIVE

This study describes the development process and design of a prototype mobile app for personalized dietary advice based on the ADG for adults in Australia, with the aim of exploring the usability of the prototype. The goal of the prototype was to provide personalized, evidence-based support for self-managing food choices in real time.

METHODS

The guidelines of the design science paradigm were applied to guide the design, development, and evaluation of a progressive web app using Amazon Web Services Elastic Compute Cloud services via iterations. The food layer of the Nutrition Care Process, the strategies of cognitive behavioral theory, and the ADG were translated into prototype features guided by the Persuasive Systems Design model. A gain-framed approach was adopted to promote positive behavior changes. A cross-modal image-to-recipe retrieval model under an Apache 2.0 license was deployed for dietary assessment. A survey using the Mobile Application Rating Scale and semistructured in-depth interviews were conducted to explore the usability of the prototype through convenience sampling (N=15).

RESULTS

The prominent features of the prototype included the use of image-based dietary assessment, food choice tracking with immediate feedback leveraging gamification principles, personal goal setting for food choices, and the provision of recipe ideas and information on the ADG. The overall prototype quality score was "acceptable," with a median of 3.46 (IQR 2.78-3.81) out of 5 points. The median score of the perceived impact of the prototype on healthy eating based on the ADG was 3.83 (IQR 2.75-4.08) out of 5 points. In-depth interviews identified the use of gamification for tracking food choices and innovation in the image-based dietary assessment as the main drivers of the positive user experience of using the prototype.

CONCLUSIONS

A novel evidence-based prototype mobile app was successfully developed by leveraging a cross-disciplinary collaboration. A detailed description of the development process and design of the prototype enhances its transparency and provides detailed insights into its creation. This study provides a valuable example of the development of a novel, evidence-based app for personalized dietary advice on food choices using recent advancements in computer vision. A revised version of this prototype is currently under development.

Assessing a nurse-assisted eHealth intervention posthospital discharge in adult patients with non-communicable diseases: a protocol for a feasibility study

INTRODUCTION

A growing number of patients with non-communicable diseases (NCDs), such as heart failure (HF) and colorectal cancer (CRC), are prone to comorbidity, a high rate of readmissions and complex healthcare needs. An eHealth intervention, however, could potentially ameliorate the increasing burdens associated with NCDs by helping to smoothen patient transition from hospital to home and by reducing the number of readmissions. This feasibility study therefore aims to assess the feasibility of a nurse-assisted eHealth intervention posthospital discharge among patients with HF and CRC, while also examining the preliminary clinical and behavioural outcomes of the intervention before initiating a full-scale randomised controlled trial. The recruitment ended in January 2023.

METHODS AND ANALYSIS

Twenty adult patients with HF and 10 adult patients with CRC will be recruited from two university hospitals in Norway. Six hospital-based nurse navigators (NNs) will offer support during the transition phase from hospital to home by using a solution for digital remote care, Dignio Connected Care. The patients will use the MyDignio application uploaded to an iPad for 30 days postdischarge. The interactions between patients and NNs will then be assessed through direct observation and qualitative interviews in line with a think-aloud protocol. Following the intervention, semistructured interviews will be used to explore patients' experiences of eHealth support and NNs' experiences of eHealth delivery. The feasibility testing will also comprise a post-test of the Post-System Usability Questionnaire and pretesting of patient-reported outcomes questionnaires, as well as an inspection of user data collected from the software.

ETHICS AND DISSEMINATION

The study has been approved by the Norwegian Centre for Research Data (ID.NO: 523386). All participation is based on informed, written consent. The results of the study will be published in open-access, peer-reviewed journals and presented at international and national scientific conferences and meetings.

Who cares for the carers? carerhelp: development and evaluation of an online resource to support the wellbeing of those caring for family members at the end of their life

BACKGROUND

Most people living with a terminal illness and approaching death will need the assistance of a non-professional carer such as a family member, friend, or neighbour to provide physical, emotional, and practical caring supports. A significant portion of these carers can feel overwhelmed, isolated and experience psychological and/or financial distress. Carers can have unmet information needs and information needs can change across the caring period.

METHODS

Guided by an Australian National Reference Group, this project undertook a multiphase set of activities to enable the development of an online carer resource. These activities included a literature review of key issues and considerations for family carers supporting someone with a terminal illness, a scoping scan of existing online resources, and interviews and focus groups with eighteen carers to understand their needs and context of caring. This information formed the basis for potential digital content. A web project team was established to create the information architecture and content pathways. User testing survey and usability assessment of the CarerHelp Website was undertaken to assess/optimise functionality prior to release. An evaluation process was also devised.

RESULTS

The literature review identified carer needs for practical and psychological support along with better education and strategies to improve communication. The scoping scan of available online resources suggested that while information available to carers is plentiful, much of that which is provided is general, disparately located, inadequately detailed, and disease specific. The eighteen carers who were interviewed highlighted the need for helpful information on: services, symptom management, relationships, preparation for death, managing the emotional and psychological burden that often accompanies caring, and support during bereavement. User testing and usability assessment of the prototype resource led to changes to enhance the user experience and effectiveness of navigation. It also highlighted a lack of awareness of existing resources and the needs of marketing and communication to address this problem.

CONCLUSIONS

The project led to the development of an open access online resource, CarerHelp ( www.carerhelp.com.au ), for use by carers and families caring for a person who has palliative care needs. The web metrics demonstrate substantial use of the resources.