Methods of usability testing in the development of eHealth applications: A scoping review

Development and validation of the mobile adherence satisfaction scale (MASS) for medication adherence apps

OBJECTIVE

To develop and validate the Mobile Adherence Satisfaction Scale (MASS) for assessing user satisfaction with mobile health applications aimed to improve medication adherence.

METHODS

The study involved patients over 18 with asthma, hypertension, heart failure, or diabetes, who used the CareAide® app for six months. Scale development included a literature review, expert consultations, and patient interviews, initially identifying 129 items. These were refined to 27 using a two-round Delphi technique and grouped into six dimensions: user interface, perceived usability, system quality, service quality, feature satisfaction, and general satisfaction. A pilot study with 30 participants further refined the model, which was then validated with 135 participants using exploratory and confirmatory factor analyses in SPSS 29 and SmartPLS 4. Data were collected via self-administered questionnaires.

RESULTS

A total of 135 complete questionnaires were analysed. Respondents had an average age of 66.7 years (SD = 11.6) with 42.2 % male (n = 57) and 57.8 % female (n = 78). After removal of an item due to cross loading, exploratory factor analysis resulted six dimensions and 26 items with Kaiser-Meyer-Olkin measure of 0.837 and Bartlett's Test of Sphericity (χ2(n = 325) = 2085.673, P < 0.001). The confirmatory factor analysis confirmed high reliability and validity: Cronbach's alpha values > 0.70 for each dimension and an overall alpha of 0.89, with Composite Reliability and Average Variance Extracted both >0.70 and >0.50, respectively, for each dimension. Structural model indicated a significant positive impact of user interface (β = 0.226, P = 0.006) and feature satisfaction (β = 0.230, P = 0.002) on general satisfaction, explaining 23.1 % of the variance (R2 = 0.231).

CONCLUSION

The study developed and validated the MASS, a reliable tool for assessing user satisfaction with mHealth apps. User interface design and feature satisfaction are key for long-term engagement and consistent medication adherence.

Supporting the Choice for Conservative and Surgical Treatment in Female Stress Urinary Incontinence: Development and Evaluation of a Patient Decision Aid

INTRODUCTION

Making a treatment decision for female stress urinary incontinence (SUI) can be challenging for patients and healthcare providers. Dutch guidelines advise to counsel both pelvic floor muscle therapy and midurethral sling surgery as primary treatment options in uncomplicated moderate to severe cases. The use of a patient decision aid (PDA) can support decision-making, reduce decisional conflict and decisional regret, and increase knowledge. The aim of this study was to develop and evaluate an online PDA for females (SUI).

METHODS

This mixed-methods study was performed in consecutive stages by a multidisciplinary working group. PDA design was based on the International Patient Decision Aids Standards (IPDAS) and on outcomes of needs assessments amongst patients and healthcare providers. Content was based on Dutch guidelines, targeted literature searches and patient information from the Dutch scientific society for gynecology. The concept version was evaluated by patients, patients' advocates, and healthcare providers.

RESULTS

Using the nominal group technique, the working group established the design and format of the PDA. Fifty-six out of 58 applicable items of the IPDAS were met. The PDA contains information on the condition, advice on lifestyle adaptations, and describes surgical and nonsurgical treatment options. The option grid contains comparisons of the primary treatment options. Furthermore, value clarification exercises and narratives were included. Acceptability and usability evaluation of the concept version was performed by 15 healthcare providers, three patients, and two patients' advocates. Comments were processed in the working group, resulting in the final version of the PDA, which was supported by all assessors.

CONCLUSION

Our multidisciplinary working group developed an online PDA for women with moderate to severe SUI including conservative and surgical treatment options, based on IPDAS criteria, guidelines, scientific evidence, and needs assessments from patients and healthcare providers. This PDA is supported by patients, healthcare providers, scientific societies, and the Dutch patients' association. The next step is to evaluate and implement this PDA in daily practice.

TRIAL REGISTRATION

ID 2014-308.

Approaches to Evaluating Digital Health Technologies: Scoping Review

BACKGROUND

Profound scientific evaluation of novel digital health technologies (DHTs) is key to enhance successful development and implementation. As such, we previously developed the eHealth evaluation cycle. The eHealth evaluation cycle contains 5 consecutive study phases: conceptual, development, feasibility, effectiveness, and implementation.

OBJECTIVE

The aim of this study is to develop a better understanding of the daily practice of the eHealth evaluation cycle. Therefore, the objectives are to conduct a structured analysis of literature data to analyze the practice of the evaluation study phases and to determine which evaluation approaches are used in which study phase of the eHealth evaluation cycle.

METHODS

We conducted a systematic literature search in PubMed including the MeSH term "telemedicine" in combination with a wide variety of evaluation approaches. Original peer-reviewed studies published in the year 2019 (pre-COVID-19 cohort) were included. Nonpatient-focused studies were excluded. Data on the following variables were extracted and systematically analyzed: journal, country, publication date, medical specialty, primary user, functionality, evaluation study phases, and evaluation approach. RStudio software was used to summarize the descriptive data and to perform statistical analyses.

RESULTS

We included 824 studies after 1583 titles and abstracts were screened. The majority of the evaluation studies focused on the effectiveness (impact; 304/824, 36.9%) study phase, whereas uptake (implementation; 70/824, 8.5%) received the least focus. Randomized controlled trials (RCTs; 170/899, 18.9%) were the most commonly used DHT evaluation method. Within the effectiveness (impact) study phase, RCTs were used in one-half of the studies. In the conceptual and planning phases, survey research (27/78, 35%) and interview studies (27/78, 35%) were most frequently used. The United States published the largest amount of DHT evaluation studies (304/824, 36.9%). Psychiatry and mental health (89/840, 10.6%) and cardiology (75/840, 8.9%) had the majority of studies published within the field.

CONCLUSIONS

We composed the first comprehensive overview of the actual practice of implementing consecutive DHT evaluation study phases. We found that the study phases of the eHealth evaluation cycle are unequally studied and most attention is paid to the effectiveness study phase. In addition, the majority of the studies used an RCT design. However, in order to successfully develop and implement novel DHTs, stimulating equal evaluation of the sequential study phases of DHTs and selecting the right evaluation approach that fits the iterative nature of technology might be of the utmost importance.

A Living Lab approach to understanding dairy farmers' technology and data needs to improve herd health: Focus groups from 6 European countries

For successful development and adoption of technology on dairy farms, farmers need to be included in the innovation process. However, the design of agricultural technologies usually takes a top-down approach with little involvement of end-users at the early stages. Living Labs offer a methodology that involve end-users throughout the development process and emphasize the importance of understanding users' needs. Currently, exploration of dairy farmers' technology needs has been limited to specific types of technology (e.g., smartphone apps) and adult cattle. The aim of this study was to use a Living Lab approach to identify dairy farmers' data and technology needs to improve herd health and inform innovation development. We conducted 18 focus groups with a total of 80 dairy farmers from Belgium, Ireland, the Netherlands, Norway, Sweden, and the United Kingdom. Data were analyzed using Template Analysis, and 6 themes were generated representing the fundamental needs of autonomy, comfort, competence, community and relatedness, purpose, and security. Farmers favored technologies that provided them with convenience, facilitated their knowledge and understanding of problems on farm, and allowed them to be self-reliant. Issues with data sharing and accessibility and usability of software were barriers to technology use. Furthermore, farmers were facing problems around recruitment and management of labor and needed ways to reduce stress. Controlling aspects of the barn environment, such as air quality, hygiene, and stocking density, were particular concerns in relation to youngstock management. Overall, the findings suggest that developers of farm technologies may want to include farmers in the design process to ensure a positive user experience and improve accessibility. The needs identified in this study can be used as a framework when designing farm technologies to strengthen need satisfaction and reduce any potential harm toward needs.

Usability testing of the Set Brave Goals app, a continence goal-selection app for children with spina bifida

INTRODUCTION

The Set Brave Goals app is the first digital health app (DHA) aimed at helping children with spina bifida (SB) aged 8-17 years old to select and track their urinary and fecal continence goals. Developed by children, parents, providers and researchers, its usability, or appropriateness to a purpose ("user-friendliness"), remains unknown. In an alpha testing stage, our aim was to determine the usability of the app prior to clinical use.

METHODS

We recruited children with SB and their parents in clinic and via social media. A recruitment goal of 10 children exceeded industry standard of 5 participants (10 participants are expected to identify approximately 96% of usability problems). Participants downloaded and used the app for a week. They completed a questionnaire, including the System Usability Scale (SUS) and closed and open-ended questions. The SUS is a 10-item validated usability questionnaire most widely used in DHA usability testing (scores range from 0 to 100, 100 representing greatest usability). Median/mean SUS scores for DHAs are 68. Mann-Whitney-U and t-tests were used.

RESULTS

Ten children with SB participated (median age: 14 years old, 6 female, 8 shunted, 8 using bladder catheterizations). Twenty parents participated (17 mothers, median age: 42 years old). Median SUS score was 77.5 for children and 73.8 for parents, corresponding to "good" to "excellent" usability (Summary Figure). There were no significant differences between child/parent scores (p = 0.69) or those for other DHAs (p = 0.11). It took a median 5-10 min to go through the app (2 parents felt it took too much time). All participants felt the app was easy to understand and use (100%). Most participants would recommend it to children and parents (children: 89%, parents: 80%), and believed it would be valuable to children (90%, 75%) and urologists (80%, 80%). Free text responses related to more varied color schemes, keeping free-text notes within the app and more flexible goal/alarm setting. These changes were incorporated into the app.

COMMENT

Usability testing prior to clinical launch identified areas for app improvement. Although this study sample met industry standards, findings are limited by a small group of participants. The app will undergo further refinement during prospective beta testing.

CONCLUSIONS

The Set Brave Goals app for children with SB has acceptable usability parameters. This justifies proceeding with wider use to formalize children's continence goal setting/tracking and testing its value to children with SB and their healthcare providers.

Requirements for a dashboard optimized for melanoma patient care through user-centered context exploration

For time-sensitive treatment of a patient with malignant melanoma, physicians must obtain a rapid overview of the patient's status. This study aimed to analyze context-specific features and processes at the point of care to derive requirements for a dashboard granting more straightforward access to information. The Think-Aloud method, contextual inquiries, and interviews were performed with physicians from the Department of Dermatology at the University Hospital Essen in Germany. The user statements and observations that were obtained were grouped and categorized using an affinity diagram. Based on the derived subjects, requirements were defined, confirmed, and prioritized. The resulting affinity diagram revealed four topics of importance at the point of care. These topics are "Identifying and Processing the Important", a comprehensive "Patient Record", tasks and challenges in the "Clinical Routine", and interactions and experiences with the available "Systems". All aspects have been reflected in 135 requirements for developing context- and indication-specific patient dashboards. Our work has elucidated the most important aspects to consider when designing a dashboard that improves patient care by enabling physicians to focus on the relevant information. Furthermore, it has been demonstrated that the aspects most often mentioned are not context-specific and can be generalized to other medical contexts.

Software Testing of eHealth Interventions: Existing Practices and the Future of an Iterative Strategy

eHealth interventions are becoming a part of standard care, with software solutions increasingly created for patients and health care providers. Testing of eHealth software is important to ensure that the software realizes its goals. Software testing, which is comprised of alpha and beta testing, is critical to establish the effectiveness and usability of the software. In this viewpoint, we explore existing practices for testing software in health care settings. We scanned the literature using search terms related to eHealth software testing (eg, "health alpha testing," "eHealth testing," and "health app usability") to identify practices for testing eHealth software. We could not identify a single standard framework for software testing in health care settings; some articles reported frameworks, while others reported none. In addition, some authors misidentified alpha testing as beta testing and vice versa. There were several different objectives (ie, testing for safety, reliability, or usability) and methods of testing (eg, questionnaires, interviews) reported. Implementation of an iterative strategy in testing can introduce flexible and rapid changes when developing eHealth software. Further investigation into the best approach for software testing in health care settings would aid the development of effective and useful eHealth software, particularly for novice eHealth software developers.

Technology Usability for People Living With Dementia: Concept Analysis

BACKGROUND

Usability is a key indicator of the quality of technology products. In tandem with technological advancements, potential use by individuals with dementia is increasing. However, defining the usability of technology for individuals with dementia remains an ongoing challenge. The diverse and progressive nature of dementia adds complexity to the creation of universal usability criteria, highlighting the need for focused deliberations. Technological interventions offer potential benefits for people living with dementia and caregivers. Amid COVID-19, technology's role in health care access is growing, especially among older adults. Enabling the diverse population of people living with dementia to enjoy the benefits of technologies requires particular attention to their needs, desires, capabilities, and vulnerabilities to potential harm from technologies. Successful technological interventions for dementia require meticulous consideration of technology usability.

OBJECTIVE

This concept analysis aims to examine the usability of technology in the context of individuals living with dementia to establish a clear definition for usability within this specific demographic.

METHODS

The framework by Walker and Avant was used to guide this concept analysis. We conducted a literature review spanning 1984 to 2024, exploring technology usability for people with dementia through the PubMed, Web of Science, and Google Scholar databases using the keywords "technology usability" and "dementia." We also incorporated clinical definitions and integrated interview data from 29 dyads comprising individuals with mild Alzheimer dementia and their respective care partners, resulting in a total of 58 older adults. This approach aimed to offer a more comprehensive portrayal of the usability needs of individuals living with dementia, emphasizing practical application.

RESULTS

The evidence from the literature review unveiled that usability encompasses attributes such as acceptable learnability, efficiency, and satisfaction. The clinical perspective on dementia stages, subtypes, and symptoms underscores the importance of tailored technology usability assessment. Feedback from 29 dyads also emphasized the value of simplicity, clear navigation, age-sensitive design, personalized features, and audio support. Thus, design should prioritize personalized assistance for individuals living with dementia, moving away from standardized technological approaches. Synthesized from various sources, the defined usability attributes for individuals living with dementia not only encompass the general usability properties of effectiveness, efficiency, and satisfaction but also include other key factors: adaptability, personalization, intuitiveness, and simplicity, to ensure that technology is supportive and yields tangible benefits for this demographic.

CONCLUSIONS

Usability is crucial for people living with dementia when designing technological interventions. It necessitates an understanding of user characteristics, dementia stages, symptoms, needs, and tasks, as well as consideration of varied physical requirements, potential sensory loss, and age-related changes. Disease progression requires adapting to evolving symptoms. Recommendations include versatile, multifunctional technology designs; accommodating diverse needs; and adjusting software functionalities for personalization. Product feature classification can be flexible based on user conditions.

A scoping review assessing the usability of digital health technologies targeting people with multiple sclerosis

Digital health technologies (DHTs) have become progressively more integrated into the healthcare of people with multiple sclerosis (MS). To ensure that DHTs meet end-users' needs, it is essential to assess their usability. The objective of this study was to determine how DHTs targeting people with MS incorporate usability characteristics into their design and/or evaluation. We conducted a scoping review of DHT studies in MS published from 2010 to the present using PubMed, Web of Science, OVID Medline, CINAHL, Embase, and medRxiv. Covidence was used to facilitate the review. We included articles that focused on people with MS and/or their caregivers, studied DHTs (including mhealth, telehealth, and wearables), and employed quantitative, qualitative, or mixed methods designs. Thirty-two studies that assessed usability were included, which represents a minority of studies (26%) that assessed DHTs in MS. The most common DHT was mobile applications (n = 23, 70%). Overall, studies were highly heterogeneous with respect to what usability principles were considered and how usability was assessed. These findings suggest that there is a major gap in the application of standardized usability assessments to DHTs in MS. Improvements in the standardization of usability assessments will have implications for the future of digital health care for people with MS.

The Solutions in Health Analytics for Rural Equity Across the Northwest (SHARE-NW) Dashboard for Health Equity in Rural Public Health: Usability Evaluation

BACKGROUND

Given the dearth of resources to support rural public health practice, the solutions in health analytics for rural equity across the northwest dashboard (SHAREdash) was created to support rural county public health departments in northwestern United States with accessible and relevant data to identify and address health disparities in their jurisdictions. To ensure the development of useful dashboards, assessment of usability should occur at multiple stages throughout the system development life cycle. SHAREdash was refined via user-centered design methods, and upon completion, it is critical to evaluate the usability of SHAREdash.

OBJECTIVE

This study aims to evaluate the usability of SHAREdash based on the system development lifecycle stage 3 evaluation goals of efficiency, satisfaction, and validity.

METHODS

Public health professionals from rural health departments from Washington, Idaho, Oregon, and Alaska were enrolled in the usability study from January to April 2022. The web-based evaluation consisted of 2 think-aloud tasks and a semistructured qualitative interview. Think-aloud tasks assessed efficiency and effectiveness, and the interview investigated satisfaction and overall usability. Verbatim transcripts from the tasks and interviews were analyzed using directed content analysis.

RESULTS

Of the 9 participants, all were female and most worked at a local health department (7/9, 78%). A mean of 10.1 (SD 1.4) clicks for task 1 (could be completed in 7 clicks) and 11.4 (SD 2.0) clicks for task 2 (could be completed in 9 clicks) were recorded. For both tasks, most participants required no prompting-89% (n=8) participants for task 1 and 67% (n=6) participants for task 2, respectively. For effectiveness, all participants were able to complete each task accurately and comprehensively. Overall, the participants were highly satisfied with the dashboard with everyone remarking on the utility of using it to support their work, particularly to compare their jurisdiction to others. Finally, half of the participants stated that the ability to share the graphs from the dashboard would be "extremely useful" for their work. The only aspect of the dashboard cited as problematic is the amount of missing data that was present, which was a constraint of the data available about rural jurisdictions.

CONCLUSIONS

Think-aloud tasks showed that the SHAREdash allows users to complete tasks efficiently. Overall, participants reported being very satisfied with the dashboard and provided multiple ways they planned to use it to support their work. The main usability issue identified was the lack of available data indicating the importance of addressing the ongoing issues of missing and fragmented public health data, particularly for rural communities.

App-based assessment of patient-reported outcomes in the Molecular Tumor Board in the Center for Personalized Medicine-(TRACE)

Background

Biomarker-based therapies are increasingly used in cancer patients outside clinical trials. Systematic assessment of patient-reported outcomes (PRO) is warranted to take patients' perspectives during biomarker-based therapies into consideration. We assessed the feasibility of an electronic PRO assessment via a smartphone application.

Methods

An interdisciplinary expert panel developed a smartphone application based on symptom burden and health-related quality of life (HRQoL) metrics reported in a retrospective analysis of 292 neuro-oncological patients. The app included validated assessments of health-related quality of life (HRQoL), the burden of symptoms, and psychological stress. Feasibility and usability were tested in a pilot study. Semi-structured interviews with patients and health care professionals (HCP) were conducted, transcribed, and analyzed according to Mayring´s qualitative content analysis. Furthermore, we assessed compliance and descriptive data of ePROs.

Results

A total of 14 patients have been enrolled, (9 female, 5 male). A total of 4 HCPs, 9 patients, and 1 caregiver were interviewed regarding usability/feasibility. The main advantages were the possibility to complete questionnaires at home and comfortable implementation in daily life. Compliance was high, for example, 82% of the weekly distributed NCCN distress thermometer questionnaires were answered on time, however, with interindividual variability. We observed a median distress score of 5 (range 0-10, 197 results, n = 12, weekly assessed) and a median Global health score of 58.3 according to the EORTC QLQ-C30 instrument (range 16.7-100, 77 results, n = 12, monthly assessed).

Conclusions

This pilot study proved the feasibility and acceptance of the app. We will therefore expand its application during biomarker-guided therapies to enable systematic PRO assessments.

Methods, Indicators, and End-User Involvement in the Evaluation of Digital Health Interventions for the Public: Scoping Review

BACKGROUND

Digital health interventions (DHIs) have the potential to enable public end users, such as citizens and patients, to manage and improve their health. Although the number of available DHIs is increasing, examples of successfully established DHIs in public health systems are limited. To counteract the nonuse of DHIs, they should be comprehensively evaluated while integrating end users. Unfortunately, there is a wide variability and heterogeneity according to the approaches of evaluation, which creates a methodological challenge.

OBJECTIVE

This scoping review aims to provide an overview of the current established processes for evaluating DHIs, including methods, indicators, and end-user involvement. The review is not limited to a specific medical field or type of DHI but offers a holistic overview.

METHODS

This scoping review was conducted following the JBI methodology for scoping reviews based on the framework by Arksey & O'Malley and complies with the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines. Three scientific databases (PubMed, Scopus, and Science Direct) were searched in April 2023. English and German studies between 2008 and 2023 were considered when evaluating DHIs that explicitly address public end users. The process of study selection was carried out by several researchers to avoid reviewer bias.

RESULTS

The search strategy identified 9618 publications, of which 160 were included. Among these included articles, 200 evaluations were derived and analyzed. The results showed that there is neither a consensus on the methods to evaluate DHIs nor a commonly agreed definition or usage of the evaluated indicators, which results in a broad variety of evaluation practices. This aligns with observations of the existing literature. It was found that there is a lack of references to existing frameworks for the evaluation of DHIs. The majority of the included studies referred to user-centered approaches and involved end users in the evaluation process. As assistance for people developing and evaluating DHIs and as a basis for thinking about appropriate ways to evaluate DHIs, a results matrix was created where the findings were combined per DHI cluster. Additionally, general recommendations for the evaluators of DHIs were formulated.

CONCLUSIONS

The findings of this scoping review offer a holistic overview of the variety and heterogeneity according to the approaches of evaluation of DHIs for public end users. Evaluators of these DHIs should be encouraged to reference established frameworks or measurements for justification. This would ease the transferability of the results among similar evaluation studies within the digital health sector, thereby enhancing the coherence and comparability of research in this area.

Evaluation of the Parkinson's Remote Interactive Monitoring System in a Clinical Setting: Usability Study

BACKGROUND

The fastest-growing neurological disorder is Parkinson disease (PD), a progressive neurodegenerative disease that affects 10 million people worldwide. PD is typically treated with levodopa, an oral pill taken to increase dopamine levels, and other dopaminergic agonists. As the disease advances, the efficacy of the drug diminishes, necessitating adjustments in treatment dosage according to the patient's symptoms and disease progression. Therefore, remote monitoring systems that can provide more detailed and accurate information on a patient's condition regularly are a valuable tool for clinicians and patients to manage their medication. The Parkinson's Remote Interactive Monitoring System (PRIMS), developed by PragmaClin Research Inc, was designed on the premise that it will be an easy-to-use digital system that can accurately capture motor and nonmotor symptoms of PD remotely.

OBJECTIVE

We performed a usability evaluation in a simulated clinical environment to assess the ease of use of the PRIMS and determine whether the product offers suitable functionality for users in a clinical setting.

METHODS

Participants were recruited from a user sign-up web-based database owned by PragmaClin Research Inc. A total of 11 participants were included in the study based on the following criteria: (1) being diagnosed with PD and (2) not being diagnosed with dementia or any other comorbidities that would make it difficult to complete the PRIMS assessment safely and independently. Patient users completed a questionnaire that is based on the Movement Disorder Society-sponsored revision of the Unified Parkinson's Disease Rating Scale. Interviews and field notes were analyzed for underlying themes and topics.

RESULTS

In total, 11 people with PD participated in the study (female individuals: n=5, 45%; male individuals: n=6, 55%; age: mean 66.7, SD 7.77 years). Thematic analysis of the observer's notes revealed 6 central usability issues associated with the PRIMS. These were the following: (1) the automated voice prompts are confusing, (2) the small camera is problematic, (3) the motor test exhibits excessive sensitivity to the participant's orientation and position in relation to the cameras, (4) the system poses mobility challenges, (5) navigating the system is difficult, and (6) the motor test exhibits inconsistencies and technical issues. Thematic analysis of qualitative interview responses revealed four central themes associated with participants' perspectives and opinions on the PRIMS, which were (1) admiration of purpose, (2) excessive system sensitivity, (3) video instructions preferred, and (4) written instructions disliked. The average system usability score was calculated to be 69.2 (SD 4.92), which failed to meet the acceptable system usability score of 70.

CONCLUSIONS

Although multiple areas of improvement were identified, most of the participants showed an affinity for the overarching objective of the PRIMS. This feedback is being used to upgrade the current PRIMS so that it aligns more with patients' needs.

Clinical Utility and Usability of the Digital Box and Block Test: Mixed Methods Study

Background

The Box and Block Test (BBT) is a clinical tool used to measure hand dexterity, which is often used for tracking disease progression or the effectiveness of therapy, particularly benefiting older adults and those with neurological conditions. Digitizing the measurement of hand function may enhance the quality of data collection. We have developed and validated a prototype that digitizes this test, known as the digital BBT (dBBT), which automatically measures time and determines and displays the test result.

Objective

This study aimed to investigate the clinical utility and usability of the newly developed dBBT and to collect suggestions for future improvements.

Methods

A total of 4 occupational therapists participated in our study. To evaluate the clinical utility, we compared the dBBT to the BBT across dimensions such as acceptance, portability, energy and effort, time, and costs. We observed therapists using the dBBT as a dexterity measurement tool and conducted a quantitative usability questionnaire using the System Usability Scale (SUS), along with a focus group. Evaluative, structured, and qualitative content analysis was used for the qualitative data, whereas quantitative analysis was applied to questionnaire data. The qualitative and quantitative data were merged and analyzed using a convergent mixed methods approach.

Results

Overall, the results of the evaluative content analysis suggested that the dBBT had a better clinical utility than the original BBT, with ratings of all collected participant statements for the dBBT being 45% (45/99) equal to, 48% (48/99) better than, and 6% (6/99) lesser than the BBT. Particularly in the subcategories "acceptance," "time required for evaluation," and "purchase costs," the dBBT was rated as being better than the original BBT. The dBBT achieved a mean SUS score of 83 (95% CI 76-96). Additionally, several suggested changes to the system were identified.

Conclusions

The study demonstrated an overall positive evaluation of the clinical utility and usability of the dBBT. Valuable insights were gathered for future system iterations. These pioneering results highlight the potential of digitizing hand dexterity assessments.

Use of the FallAkte Plus System as an IT Infrastructure for the North Rhine-Westphalian General Practice Research Network: Mixed Methods Usability Study

BACKGROUND

Primary care research networks can generate important information in the setting where most patients are seen and treated. However, this requires a suitable IT infrastructure (ITI), which the North Rhine-Westphalian general practice research network is looking to implement.

OBJECTIVE

This mixed methods research study aims to evaluate (study 1) requirements for an ITI and (study 2) the usability of an IT solution already available on the market, the FallAkte Plus (FA+) system for the North Rhine-Westphalian general practice research network, which comprises 8 primary care university institutes in Germany's largest state.

METHODS

In study 1, a survey was conducted among researchers from the institutes to identify the requirements for a suitable ITI. The questionnaire consisted of standardized questions with open-ended responses. In study 2, a mixed method approach combining a think-aloud approach and a quantitative survey was used to evaluate the usability and acceptance of the FA+ system among 3 user groups: researchers, general practitioners, and practice assistants. Respondents were asked to assess the usability with the validated system usability scale and to test a short questionnaire on vaccination management through FA+.

RESULTS

In study 1, five of 8 institutes participated in the requirements survey. A total of 32 user requirements related primarily to study management were identified, including data entry, data storage, and user access management. In study 2, a total of 36 participants (24 researchers and 12 general practitioners or practice assistants) were surveyed in the mixed methods study of an already existing IT solution. The tutorial video and handouts explaining how to use the FA+ system were well received. Researchers, unlike practice personnel, were concerned about data security and data protection regarding the system's emergency feature, which enables access to all patient data. The median overall system usability scale rating was 60 (IQR 33.0-85.0), whereby practice personnel (median 82, IQR 58.0-94.0) assigned higher ratings than researchers (median 44, IQR 14.0-61.5). Users appreciated the option to integrate data from practices and other health care facilities. However, they voted against the use of the FA+ system due to a lack of support for various study formats.

CONCLUSIONS

Usability assessments vary markedly by professional group and role. In its current stage of development, the FA+ system does not fully meet the requirements for a suitable ITI. Improvements in the user interface, performance, interoperability, security, and advanced features are necessary to make it more effective and user-friendly. Collaborating with end users and incorporating their feedback are crucial for the successful development of any practice network research ITI.

The impact of a mixed reality technology-driven health enhancing physical activity program among community-dwelling older adults: a study protocol

Background

Physical inactivity and a sedentary lifestyle among community-dwelling older adults poses a greater risk for progressive physical and cognitive decline. Mixed reality technology-driven health enhancing physical activities such as the use of virtual coaches provide an emerging and promising solution to support healthy lifestyle, but the impact has not been clearly understood.

Methods and analysis

An observational explanatory sequential mixed-method research design was conceptualized to examine the potential impact of a user-preferred mixed reality technology-driven health enhancing physical activity program directed toward purposively selected community-dwelling older adults in two senior centers in the Philippines. Quantitative components of the study will be done through a discreet choice experiment and a quasi-experimental study. A total of 128, or 64 older adults in each center, will be recruited via posters at community senior centers who will undergo additional screening or health records review by a certified gerontologist to ensure safety and proper fit. Treatments (live coaching with video-based exercise and mixed reality technology-driven exercise) will be assigned to each of the two senior center sites for the quasi-experiment. The participants from the experimental group shall be involved in the discreet choice experiment, modeling, and usability evaluations. Finally, a qualitative sample of participants (n = 6) as key informants shall be obtained from the experimental group using purposive selection.

Discussion

This study protocol will examine the health impact of a promising mixed reality program in health promotion among older adults. The study utilizes a human-centered mixed method research design in technology development and evaluation in the context of developing nations.Clinical trial registration: ClinicalTrials.gov, identifier NCT06136468.

Evaluation of a Digital Amsler Grid (PocDoc) for Macular Disease Screening: A Comparative Analysis with the Conventional Method

INTRODUCTION

Macular diseases are major contributors to visual impairment and blindness worldwide. This study introduces PocDoc, a digital version of the conventional Amsler grid, aimed at enhancing the screening and monitoring of macular diseases. We conducted a comprehensive evaluation to compare the effectiveness of PocDoc against the conventional method.

METHODS

Our comparative analysis involved two distinct phases. Initially, we assessed the capability of both PocDoc and the conventional method in detecting central visual field abnormalities. This phase included a cohort of 72 healthy and 155 eyes affected by various conditions such as age-related macular degeneration (AMD), uveitis, polypoidal choroidal vasculopathy (PCV), and macular telangiectasia. We primarily focused on the area of compromise and observed the correlation between the results obtained from both methods, measuring their concordance using a correlation coefficient. In the second phase, we evaluated the accuracy of both methods in diagnosing AMD. This involved a group of 127 eyes, including 70 healthy and 57 AMD-affected eyes. We determined the sensitivity, specificity, and overall accuracy of each method in diagnosing AMD.

RESULTS

In the initial phase, both PocDoc and the conventional Amsler grid demonstrated a high correlation in detecting central visual field defects across various macular diseases (correlation coefficient > 0.9). In the second phase, focused on AMD diagnosis, PocDoc showed a sensitivity of 50%, specificity of 100%, and an overall accuracy of 78%. Comparatively, the conventional method exhibited a sensitivity of 49%, specificity of 100%, and accuracy of 77%.

CONCLUSION

PocDoc's digital Amsler grid exhibits comparable effectiveness to the conventional method in both detecting visual field abnormalities across a range of macular diseases and specifically in the diagnosis of AMD. The high correlation in results, combined with the digital advantages of PocDoc, such as ease of use and potential for telemedicine applications, suggests its viability as a valuable tool in the screening and monitoring of macular diseases.

Automated Personalized Self-care Program for Patients With Type 2 Diabetes Mellitus: A Pilot Trial

PURPOSE

Providing continuous self-care support to the growing diabetes population is challenging. Strategies are needed to enhance engagement in self-care, utilizing innovative technologies for personalized feedback. This study aimed to assess the feasibility of the Automated Personalized Self-Care program among type 2 diabetes patients and evaluate its preliminary effectiveness.

METHODS

A parallel randomized pilot trial with qualitative interviews occurred from May 3, 2022, to September 27, 2022. Participants aged 40-69 years with type 2 diabetes and HbA1c ≥ 7.0% were recruited. The three-month program involved automated personalized goal setting, education, monitoring, and feedback. Feasibility was measured by participants' engagement and intervention usability. Preliminary effectiveness was examined through self-care self-efficacy, self-care behaviors, and health outcomes. Qualitative interviews were conducted with the intervention group.

RESULTS

A total of 404 patients were screened. Out of the 61 eligible patients, 32 were enrolled, resulting in a recruitment rate of 52.5%. Retention rates at three months were 84.2% and 84.6% in the intervention and control groups, respectively. Among the intervention group, 81.3% satisfied adherence criteria. Mobile application's usability scored 66.25, and participants' satisfaction was 8.06. Intention-to-treat analysis showed improvements in self-measured blood glucose testing, grain intake, and HbA1c in the intervention group. Qualitative content analysis identified nine themes.

CONCLUSION

Feasibility of the program was verified. A larger randomized controlled trial is needed to determine its effectiveness in self-care self-efficacy, self-care behaviors, and health outcomes among type 2 diabetes patients. This study offers insights for optimizing future trials assessing clinical effectiveness.

TRIAL REGISTRATION

Clinical Research Information Service, KCT0008202 (registration date: 17 February 2023).

Conceptually Funded Usability Evaluation of an Application for Leveraging Descriptive Data Analysis Models for Cardiovascular Research

The focus of this study, and the subject of this article, resides in the conceptually funded usability evaluation of an application of descriptive models to a specific dataset obtained from the East Slovak Institute of Heart and Vascular Diseases targeting cardiovascular patients. Delving into the current state-of-the-art practices, we examine the extent of cardiovascular diseases, descriptive data analysis models, and their practical applications. Most importantly, our inquiry focuses on exploration of usability, encompassing its application and evaluation methodologies, including Van Welie's layered model of usability and its inherent advantages and limitations. The primary objective of our research was to conceptualize, develop, and validate the usability of an application tailored to supporting cardiologists' research through descriptive modeling. Using the R programming language, we engineered a Shiny dashboard application named DESSFOCA (Decision Support System For Cardiologists) that is structured around three core functionalities: discovering association rules, applying clustering methods, and identifying association rules within predefined clusters. To assess the usability of DESSFOCA, we employed the System Usability Scale (SUS) and conducted a comprehensive evaluation. Additionally, we proposed an extension to Van Welie's layered model of usability, incorporating several crucial aspects deemed essential. Subsequently, we rigorously evaluated the proposed extension within the DESSFOCA application with respect to the extended usability model, drawing insightful conclusions from our findings.

Assessing Differences in mHealth Usability and App Experiences Among Young African American Women: Secondary Analysis of a Randomized Controlled Trial

BACKGROUND

In North Carolina, HIV continues to disproportionately affect young African American women. Although mobile health (mHealth) technology appears to be a tool capable of making public health information more accessible for key populations, previous technology use and social determinants may impact users' mHealth experiences.

OBJECTIVE

The objective of this study was to evaluate mHealth usability, assessing differences based on previous technology use and social determinants among a sample of African American women in emerging adulthood.

METHODS

As part of a National Institute on Drug Abuse-funded randomized controlled trial with African American women (aged 18-25 years), counties were assigned to receive an evidence-based HIV risk reduction intervention through mHealth and participants were asked to complete usability surveys at 6- and 12-month follow-ups. Participants' first survey responses were analyzed through 2-tailed t tests and linear regression models to examine associations with previous technology use and social determinants (P<.05).

RESULTS

The mean System Usability Scale (SUS) score was 69.2 (SD 17.9; n=159), which was higher than the threshold of acceptability (68.0). Participants who had previously used a tablet indicated higher usability compared to participants without previous use (mean 72.9, SD 18.1 vs mean 57.6, SD 11.4; P<.001), and participants with previous smartphone use also reported higher usability compared to participants without previous use (mean 71.9, SD 18.3 vs mean 58.0, SD 10.7; P<.001). Differences in SUS scores were observed among those reporting homelessness (mean 58.3, SD 19.0 vs mean 70.8, SD 17.2; P=.01), unemployment (mean 65.9, SD 17.2 vs mean 71.6, SD 18.1; P=.04), or current school enrollment (mean 73.2, SD 18.5 vs mean 65.4, SD 16.5; P=.006). Statistically significant associations were not observed for food insecurity (mean 67.3, SD 18.6 vs mean 69.9, SD 17.7; P=.45).

CONCLUSIONS

Although above-average usability was observed overall, these findings demonstrate differences in mHealth usability based on past and current life experiences. As mHealth interventions become more prevalent, these findings may have important implications for ensuring that mHealth apps improve the reach of evidence-based interventions.

TRIAL REGISTRATION

ClinicalTrials.gov NCT02965014; https://clinicaltrials.gov/study/NCT02965014.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.1186/s12889-018-5796-8.

ChatGPT-3.5 System Usability Scale early assessment among Healthcare Workers: Horizons of adoption in medical practice

Artificial intelligence (AI) chatbots, such as ChatGPT, have widely invaded all domains of human life. They have the potential to transform healthcare future. However, their effective implementation hinges on healthcare workers' (HCWs) adoption and perceptions. This study aimed to evaluate HCWs usability of ChatGPT three months post-launch in Saudi Arabia using the System Usability Scale (SUS). A total of 194 HCWs participated in the survey. Forty-seven percent were satisfied with their usage, 57 % expressed moderate to high trust in its ability to generate medical decisions. 58 % expected ChatGPT would improve patients' outcomes, even though 84 % were optimistic of its potential to improve the future of healthcare practice. They expressed possible concerns like recommending harmful medical decisions and medicolegal implications. The overall mean SUS score was 64.52, equivalent to 50 % percentile rank, indicating high marginal acceptability of the system. The strongest positive predictors of high SUS scores were participants' belief in AI chatbot's benefits in medical research, self-rated familiarity with ChatGPT and self-rated computer skills proficiency. Participants' learnability and ease of use score correlated positively but weakly. On the other hand, medical students and interns had significantly high learnability scores compared to others, while ease of use scores correlated very strongly with participants' perception of positive impact of ChatGPT on the future of healthcare practice. Our findings highlight the HCWs' perceived marginal acceptance of ChatGPT at the current stage and their optimism of its potential in supporting them in future practice, especially in the research domain, in addition to humble ambition of its potential to improve patients' outcomes particularly in regard of medical decisions. On the other end, it underscores the need for ongoing efforts to build trust and address ethical and legal concerns of AI implications in healthcare. The study contributes to the growing body of literature on AI chatbots in healthcare, especially addressing its future improvement strategies and provides insights for policymakers and healthcare providers about the potential benefits and challenges of implementing them in their practice.

Empowering patients in primary care: a qualitative exploration of the usability and utility of an online diabetes self-management tool

BACKGROUND

Despite the potential advantages of Internet-based diabetes self-management education, its adoption was not widespread among Singapore's public primary care clinics (polyclinics). An interactive online tool was thus developed to help educate patients with Type 2 diabetes mellitus (T2DM), and was now ready for user testing before implementation.

AIM

To explore the perceived utility and usability of the educational tool in patients with suboptimally-controlled T2DM in a Singapore primary care setting.

METHODS

In-depth interviews were used to gather qualitative data from multi-ethnic Asian adults who had suboptimally-controlled T2DM. A total of 17 IDIs were conducted between April 2022 to March 2023, audio-recorded, transcribed, and analyzed to identify emergent themes via thematic analysis.

RESULTS

Regarding utility, users found the educational tool useful because it provided them with information that was comprehensive, accessible, reliable, and manageable. Regarding usability, the majority of users reported that the educational tool was easy to use, and suggested ways to improve navigational cues, visual clarity, readability and user engagement.

CONCLUSION

Participants generally found the educational tool useful and easy to use. A revised educational tool will be developed based on their feedback and implemented in clinical practice.

Usability of an Automated System for Real-Time Monitoring of Shared Decision-Making for Surgery: Mixed Methods Evaluation

BACKGROUND

Improving shared decision-making (SDM) for patients has become a health policy priority in many countries. Achieving high-quality SDM is particularly important for approximately 313 million surgical treatment decisions patients make globally every year. Large-scale monitoring of surgical patients' experience of SDM in real time is needed to identify the failings of SDM before surgery is performed. We developed a novel approach to automating real-time data collection using an electronic measurement system to address this. Examining usability will facilitate its optimization and wider implementation to inform interventions aimed at improving SDM.

OBJECTIVE

This study examined the usability of an electronic real-time measurement system to monitor surgical patients' experience of SDM. We aimed to evaluate the metrics and indicators relevant to system effectiveness, system efficiency, and user satisfaction.

METHODS

We performed a mixed methods usability evaluation using multiple participant cohorts. The measurement system was implemented in a large UK hospital to measure patients' experience of SDM electronically before surgery using 2 validated measures (CollaboRATE and SDM-Q-9). Quantitative data (collected between April 1 and December 31, 2021) provided measurement system metrics to assess system effectiveness and efficiency. We included adult patients booked for urgent and elective surgery across 7 specialties and excluded patients without the capacity to consent for medical procedures, those without access to an internet-enabled device, and those undergoing emergency or endoscopic procedures. Additional groups of service users (group 1: public members who had not engaged with the system; group 2: a subset of patients who completed the measurement system) completed user-testing sessions and semistructured interviews to assess system effectiveness and user satisfaction. We conducted quantitative data analysis using descriptive statistics and calculated the task completion rate and survey response rate (system effectiveness) as well as the task completion time, task efficiency, and relative efficiency (system efficiency). Qualitative thematic analysis identified indicators of and barriers to good usability (user satisfaction).

RESULTS

A total of 2254 completed surveys were returned to the measurement system. A total of 25 service users (group 1: n=9; group 2: n=16) participated in user-testing sessions and interviews. The task completion rate was high (169/171, 98.8%) and the survey response rate was good (2254/5794, 38.9%). The median task completion time was 3 (IQR 2-13) minutes, suggesting good system efficiency and effectiveness. The qualitative findings emphasized good user satisfaction. The identified themes suggested that the measurement system is acceptable, easy to use, and easy to access. Service users identified potential barriers and solutions to acceptability and ease of access.

CONCLUSIONS

A mixed methods evaluation of an electronic measurement system for automated, real-time monitoring of patients' experience of SDM showed that usability among patients was high. Future pilot work will optimize the system for wider implementation to ultimately inform intervention development to improve SDM.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.1136/bmjopen-2023-079155.

A Technological Tool Aimed at Self-Care in Patients With Multimorbidity: Cross-Sectional Usability Study

BACKGROUND

Information and communication technologies (ICTs) have been positioned as useful tools to facilitate self-care. The interaction between a patient and technology, known as usability, is particularly important for achieving positive health outcomes. Specific characteristics of patients with chronic diseases, including multimorbidity, can affect their interaction with different technologies. Thus, studying the usability of ICTs in the field of multimorbidity has become a key element to ensure their relevant role in promoting self-care.

OBJECTIVE

The aim of this study was to analyze the usability of a technological tool dedicated to health and self-care in patients with multimorbidity in primary care.

METHODS

A descriptive observational cross-sectional usability study was performed framed in the clinical trial in the primary care health centers of Madrid Health Service of the TeNDER (Affective Based Integrated Care for Better Quality of Life) project. The TeNDER technological tool integrates sensors for monitoring physical and sleep activity along with a mobile app for consulting the data collected and working with self-management tools. This project included patients over 60 years of age who had one or more chronic diseases, at least one of which was mild-moderate cognitive impairment, Parkinson disease, or cardiovascular disease. From the 250 patients included in the project, 38 agreed to participate in the usability study. The usability variables investigated were effectiveness, which was determined by the degree of completion and the total number of errors per task; efficiency, evaluated as the average time to perform each task; and satisfaction, quantified by the System Usability Scale. Five tasks were evaluated based on real case scenarios. Usability variables were analyzed according to the sociodemographic and clinical characteristics of patients. A logistic regression model was constructed to estimate the factors associated with the type of support provided for task completion.

RESULTS

The median age of the 38 participants was 75 (IQR 72.0-79.0) years. There was a slight majority of women (20/38, 52.6%) and the participants had a median of 8 (IQR 7.0-11.0) chronic diseases. Thirty patients completed the usability study, with a usability effectiveness result of 89.3% (134/150 tasks completed). Among the 30 patients, 66.7% (n=20) completed all tasks and 56.7% (17/30) required personalized help on at least one task. In the multivariate analysis, educational level emerged as a facilitating factor for independent task completion (odds ratio 1.79, 95% CI 0.47-6.83). The median time to complete the total tasks was 296 seconds (IQR 210.0-397.0) and the median satisfaction score was 55 (IQR 45.0-62.5) out of 100.

CONCLUSIONS

Although usability effectiveness was high, the poor efficiency and usability satisfaction scores suggest that there are other factors that may interfere with the results. Multimorbidity was not confirmed to be a key factor affecting the usability of the technological tool.

TRIAL REGISTRATION

Clinicaltrials.gov NCT05681065; https://clinicaltrials.gov/study/NCT05681065.

Designing for usability: development and evaluation of a portable minimally-actuated haptic hand and forearm trainer for unsupervised stroke rehabilitation

In stroke rehabilitation, simple robotic devices hold the potential to increase the training dosage in group therapies and to enable continued therapy at home after hospital discharge. However, we identified a lack of portable and cost-effective devices that not only focus on improving motor functions but also address sensory deficits. Thus, we designed a minimally-actuated hand training device that incorporates active grasping movements and passive pronosupination, complemented by a rehabilitative game with meaningful haptic feedback. Following a human-centered design approach, we conducted a usability study with 13 healthy participants, including three therapists. In a simulated unsupervised environment, the naive participants had to set up and use the device based on written instructions. Our mixed-methods approach included quantitative data from performance metrics, standardized questionnaires, and eye tracking, alongside qualitative feedback from semi-structured interviews. The study results highlighted the device's overall ease of setup and use, as well as its realistic haptic feedback. The eye-tracking analysis further suggested that participants felt safe during usage. Moreover, the study provided crucial insights for future improvements such as a more intuitive and comfortable wrist fixation, more natural pronosupination movements, and easier-to-follow instructions. Our research underscores the importance of continuous testing in the development process and offers significant contributions to the design of user-friendly, unsupervised neurorehabilitation technologies to improve sensorimotor stroke rehabilitation.

Evaluating the Usability of an mHealth App for Empowering Cancer Survivors With Disabilities: Heuristic Evaluation and Usability Testing

BACKGROUND

More than 18 million cancer survivors are living in the United States. The effects of cancer and its treatments can have cognitive, psychological, physical, and social consequences that many survivors find incredibly disabling. Posttreatment support is often unavailable or underused, especially for survivors living with disabilities. This leaves them to deal with new obstacles and struggles on their own, oftentimes feeling lost during this transition. Mobile health (mHealth) interventions have been shown to effectively aid cancer survivors in dealing with many of the aftereffects of cancer and its treatments; these interventions hold immense potential for survivors living with disabilities. We developed a prototype for WeCanManage, an mHealth-delivered self-management intervention to empower cancer survivors living with disabilities through problem-solving, mindfulness, and self-advocacy training.

OBJECTIVE

Our study conducted a heuristic evaluation of the WeCanManage high-fidelity prototype and assessed its usability among cancer survivors with known disabilities.

METHODS

We evaluated the prototype using Nielsen's 10 principles of heuristic evaluation with 22 human-computer interaction university students. On the basis of the heuristic evaluation findings, we modified the prototype and conducted usability testing on 10 cancer survivors with a variety of known disabilities, examining effectiveness, efficiency, usability, and satisfaction, including a completion of the modified System Usability Scale (SUS).

RESULTS

The findings from the heuristic evaluation were mostly favorable, highlighting the need for a help guide, addressing accessibility concerns, and enhancing the navigation experience. After usability testing, the average SUS score was 81, indicating a good-excellent design. The participants in the usability testing sample expressed positive reactions toward the app's design, educational content and videos, and the available means of connecting with others. They identified areas for improvement, such as improving accessibility, simplifying navigation within the community forums, and providing a more convenient method to access the help guide.

CONCLUSIONS

Overall, usability testing showed positive results for the design of WeCanManage. The course content and features helped participants feel heard, understood, and less alone.

Feasibility, accuracy, and usability analysis of MapAML, a first-in-class app for integrated diagnosis in acute myeloid leukemia

Performing a comprehensive diagnosis of acute myeloid leukemia (AML) is complex and involves the integration of clinical information, bone marrow morphology, immunophenotyping, cytogenetic, and molecular analysis, which can be challenging to the general hematologist. The aim of this study was to evaluate the usability and accuracy of MapAML, a smartphone app for integrated diagnosis in AML, created to aid the hematologist in its clinical practice. App performance was evaluated in dedicated sessions, in which 21 hematologists or fellows in hematology performed an integrated diagnosis of deidentified real-world clinical AML cases, first without and posteriorly with MapAML use. Diagnosis accuracy increased after MapAML utilization, with the average score going from 7.08 without app to 8.88 with app use (on a scale from 0 to 10), representing a significant accuracy improvement (p = .002). Usability evaluation was very favorable, with 81% of users considering the app very or extremely simple to use. There was also a significant increase in confidence to perform a complete and accurate diagnosis in AML after app use, with 61.9% of the participants willing to use the app in their clinical practice. In this study, MapAML increased accuracy with excellent usability for integrated diagnosis in AML.

Development of oral health resources and a mobile app for caregivers and autistic children through consensus building

LAY ABSTRACT

Caregivers of autistic children often lack knowledge regarding oral homecare and when and where to see the dentist. To address this need, we developed a series of information on oral health. An autistic child assisted in developing two social stories to showcase a dental visit. A mobile app was developed to deliver the above mentioned. Other features include creation of customised social stories and visual schedule and an inbox to allow dentists to send messages to parents. The developed information and social stories were reviewed by experts and parents. The app also underwent anonymous and independent testing by parents. Overall the information and app were well received by the experts and parents.

Comparison of evaluation methods for improving the usability of a Spanish mHealth tool

OBJECTIVE

Mobile health (mHealth) technology is now widely used across health conditions and populations. The rigorous development of these tools has yielded improved health outcomes, yet the ideal approach for developing mHealth tools continues to evolve, indicating the need for rigorous usability evaluation methods. This study compares two usability evaluation methods - cognitive interviews and usability assessments employing a think-aloud approach - for adapting an evidence-based mHealth tool from English into Spanish.

METHODS

We conducted cognitive interviews and usability assessments using a think-aloud protocol to evaluate the usability of an HIV mHealth application among 40 Spanish-speaking adults with HIV in New York City, NY, and La Romana, Dominican Republic. The Health IT Usability Evaluation Model (Health-ITUEM) was used to guide the analysis of qualitative data collected from each method.

RESULTS

Participants (N = 40) averaged 43 years old (SD = 12.26; range 20-79), identified primarily Hispanic/Latino (92.5 %), and resided in La Romana (50 %) or New York City (50 %). Both usability evaluation methods yielded similar findings, highlighting learnability and information needs as crucial components of participant feedback for the mHealth application. Cognitive interviews captured participants' perspectives on the app's interface and design. On the other hand, results from usability assessments offered insights into participants' competency while interacting with the mHealth tool.

CONCLUSION

Findings from this study highlight the contributions and limitations of including cognitive interviews and task-based usability assessments using a think-aloud approach in mHealth usability testing. Future research should employ a multi-method approach, incorporating complementary usability evaluation methods and engaging participants in multiple assessments. Using complementary usability evaluation methods may provide a more comprehensive understanding of the usability and participant experience aspects of a mHealth tool compared to using a single usability evaluation method.

World of sounds (Seslerin Dünyası): A mobile auditory training game for children with cochlear implants

OBJECTIVES

The aim of this study is to develop a mobile auditory training application based on gaming for children aged 3-5 years using cochlear implants and to evaluate its usability.

METHODS

4 games were developed in the scope of the application World of Sounds; the crucible sound for auditory awareness, mole hunting for auditory discrimination, find the sound for auditory recognition, and choo-choo for auditory comprehension. The prototype was applied to 20 children with normal hearing and 20 children with cochlear implants, all of whom were aged 3-5. The participants were asked to fill out the Game Evaluation Form for Children. Moreover, 40 parents were included in the study, and the Evaluation Form for the Application was applied.

RESULTS

According to the form, at least 80% of children using cochlear implants, and all children in the healthy group, responded well to the usability factors. All factors were obtained as highly useable by parents of the children using cochlear implants. The results indicated that in the healthy group, the usefulness and motivation factors were above moderate, and the other factors were highly useable. In the mole-hunting game, there was no significant difference between the groups in the easy level of the first sub-section (p > 0.05). There was a significant difference between the groups in terms of the other sub-sections of the mole-hunting game and all sub-sections of the crucible sound, find the sound, and the choo-choo games (p < 0.05). While there was no correlation between duration of cochlear implant use and ADSI scores and the third sub-section of the crucible sound game (p > 0.05); a correlation was found in the other sub-sections of crucible sound and all sub-sections of the mole hunting, find the sound, and Choo-Choo games (p < 0.05).

CONCLUSION

It is thought that the application World of Sounds can serve as an accessible option to support traditional auditory rehabilitation for children with cochlear implants.

WeChat mini program in laboratory biosafety education among medical students at Guangzhou Medical University: a mixed method study of feasibility and usability

BACKGROUND

Laboratory biosafety should be a priority in all healthcare institutions. In traditional laboratory safety teaching students typically receive knowledge passively from their teachers without active involvement. The combination of experiential learning and mobile learning may provide students with greater engagement, retention, and application of knowledge. To address this issue, we developed and conducted a convergent mixed methods study to assess the feasibility and usability of a WeChat mini program (WMP) named WeMed for laboratory biosafety education for medical laboratory students at Guangzhou Medical University (GMU).

METHODS

The study was conducted between November 2022 and October 2023 among second-year undergraduate students at GMU. It involved the concurrent collection, analysis, and interpretation of both qualitative and quantitative data to assess feasibility and usability. In the quantitative strand, two evaluations were conducted via online surveys from students (n = 67) after a four-week study period. The System Usability Scale (SUS) was used to evaluate usability, while self-developed questions were used to assess feasibility. Additionally, a knowledge test was administered 6 months after the program completion. In the qualitative strand, fourteen semi-structured interviews were conducted, whereby a reflexive thematic analysis was utilized to analyze the interview data.

RESULTS

The overall SUS score is adequate (M = 68.17, SD = 14.39). The acceptability of the WeMed program is in the marginal high range. Most students agreed that WeMed was useful for learning biosafety knowledge and skills (13/14, 93%), while 79% (11/14) agreed it was easy to use and they intended to continue using it. After 6 months, a significant difference in the knowledge test scores was observed between the WeMed group (n = 67; 2nd year students) and the traditional training group (n = 90; 3rd year students). However, the results should be interpreted cautiously due to the absence of a pretest.

CONCLUSION

The combination of experiential learning and mobile learning with WMP is a feasible tool for providing laboratory biosafety knowledge and skills. Ongoing improvements should be made in order to increase long-term acceptance.

Testing Two Online Symptom Checkers With Vulnerable Groups: Usability Study to Improve Cognitive Accessibility of eHealth Services

BACKGROUND

The popularity of eHealth services has surged significantly, underscoring the importance of ensuring their usability and accessibility for users with diverse needs, characteristics, and capabilities. These services can pose cognitive demands, especially for individuals who are unwell, fatigued, or experiencing distress. Additionally, numerous potentially vulnerable groups, including older adults, are susceptible to digital exclusion and may encounter cognitive limitations related to perception, attention, memory, and language comprehension. Regrettably, many studies overlook the preferences and needs of user groups likely to encounter challenges associated with these cognitive aspects.

OBJECTIVE

This study primarily aims to gain a deeper understanding of cognitive accessibility in the practical context of eHealth services. Additionally, we aimed to identify the specific challenges that vulnerable groups encounter when using eHealth services and determine key considerations for testing these services with such groups.

METHODS

As a case study of eHealth services, we conducted qualitative usability testing on 2 online symptom checkers used in Finnish public primary care. A total of 13 participants from 3 distinct groups participated in the study: older adults, individuals with mild intellectual disabilities, and nonnative Finnish speakers. The primary research methods used were the thinking-aloud method, questionnaires, and semistructured interviews.

RESULTS

We found that potentially vulnerable groups encountered numerous issues with the tested services, with similar problems observed across all 3 groups. Specifically, clarity and the use of terminology posed significant challenges. The services overwhelmed users with excessive information and choices, while the terminology consisted of numerous complex medical terms that were difficult to understand. When conducting tests with vulnerable groups, it is crucial to carefully plan the sessions to avoid being overly lengthy, as these users often require more time to complete tasks. Additionally, testing with vulnerable groups proved to be quite efficient, with results likely to benefit a wider audience as well.

CONCLUSIONS

Based on the findings of this study, it is evident that older adults, individuals with mild intellectual disability, and nonnative speakers may encounter cognitive challenges when using eHealth services, which can impede or slow down their use and make the services more difficult to navigate. In the worst-case scenario, these challenges may lead to errors in using the services. We recommend expanding the scope of testing to include a broader range of eHealth services with vulnerable groups, incorporating users with diverse characteristics and capabilities who are likely to encounter difficulties in cognitive accessibility.

Digital clinical empathy in a live chat: multiple findings from a formative qualitative study and usability tests

BACKGROUND

Clinical empathy is considered a crucial element in patient-centered care. The advent of digital technology in healthcare has introduced new dynamics to empathy which needs to be explored in the context of the technology, particularly within the context of written live chats. Given the growing prevalence of written live chats, this study aimed to explore and evaluate techniques of digital clinical empathy within a familial cancer-focused live chat, focusing on how health professionals can (a) understand, (b) communicate, and (c) act upon users' perspectives and emotional states.

METHODS

The study utilized a qualitative approach in two research phases. It examined the expected and implemented techniques and effectiveness of digital clinical empathy in a live chat service, involving semi-structured interviews with health professionals (n = 9), focus group discussions with potential users (n = 42), and two rounds of usability tests between health professionals (n = 9) and users (n = 18). Data were examined using qualitative content analysis.

RESULTS

Expected techniques of digital clinical empathy, as articulated by both users and health professionals, involve reciprocal engagement, timely responses, genuine authenticity, and a balance between professionalism and informality, all while going beyond immediate queries to facilitate informed decision-making. Usability tests confirm these complexities and introduce new challenges, such as balancing timely, authentic responses with effective, personalized information management and carefully framed referrals.

CONCLUSIONS

The study reveals that the digital realm adds layers of complexity to the practice of clinical empathy. It underscores the importance of ongoing adaptation and suggests that future developments could benefit from a hybrid model that integrates the strengths of both AI and human health professionals to meet evolving user needs and maintain high-quality, empathetic healthcare interactions.

Usability and Evaluation of a Health Information System in the Emergency Department: Mixed Methods Study

BACKGROUND

A lack of information during an emergency visit leads to the experience of powerlessness for patients and their family members, who may also feel unprepared to cope with acute symptoms. The ever-changing nature and fast-paced workflow in the emergency department (ED) often affect how health care professionals can tailor information and communication to the needs of the patient.

OBJECTIVE

This study aimed to evaluate the usability and experience of a newly developed information system. The system was developed together with patients and their family members to help provide the information needed in the ED.

METHODS

We conducted a mixed methods study consisting of quantitative data obtained from the System Usability Scale questionnaire and qualitative interview data obtained from purposively selected participants included in the quantitative part of the study.

RESULTS

A total of 106 patients and 14 family members (N=120) answered the questionnaire. A total of 10 patients and 3 family members participated in the interviews. Based on the System Usability Scale score, the information system was rated close to excellent, with a mean score of 83.6 (SD 12.8). Most of the participants found the information system easy to use and would like to use it again. The participants reported that the system helped them feel in control, and the information was useful. Simplifications were needed to improve the user experience for the older individuals.

CONCLUSIONS

This study demonstrates that the usability of the information system is rated close to excellent. It was perceived to be useful as it enabled understanding and predictability of the patient's trajectory in the ED. Areas for improvement include making the system more usable by older individuals. The study provides an example of how a technological solution can be used to diminish the information gap in an ED context.

Web-Based Screening, Brief Intervention, and Referral to Treatment for Traumatic Stress and Alcohol Misuse Among Survivors of Sexual Assault and Intimate Partner Violence: Usability and Acceptability Study

BACKGROUND

Recent survivors of intimate partner violence (IPV) and sexual assault (SA) are at a high risk for traumatic stress and alcohol misuse. IPV and SA survivors face barriers to services for traumatic stress and alcohol misuse and have low service utilization rates. One way to increase access to services for this population is the use of web-based screening, brief intervention, and referral to treatment (SBIRT), an evidence-informed approach for early identification of traumatic stress and alcohol and drug misuse and connecting individuals to treatment.

OBJECTIVE

This study aims to assess the usability and acceptability of a web-based SBIRT called CHAT (Choices For Your Health After Trauma) tailored to address traumatic stress and alcohol misuse following past-year IPV, SA, or both.

METHODS

Phase 1 involved gathering feedback about usability and acceptability from focus groups with victim service professionals (22/52, 42%) and interviews with past-year survivors of IPV, SA, or both (13/52, 25%). Phase 2 involved gathering feedback about the acceptability of an adapted version of CHAT in an additional sample of recent survivors (17/52, 33%). Survey data on history of IPV and SA, posttraumatic stress disorder symptoms, alcohol and drug use, and service use were collected from survivors in both phases to characterize the samples. Qualitative content and thematic analyses of the interviews and focus group data were conducted using a coding template analysis comprising 6 a priori themes (usability, visual design, user engagement, content, therapeutic persuasiveness, and therapeutic alliance).

RESULTS

Six themes emerged during the focus groups and interviews related to CHAT: usability, visual design, user engagement, content, therapeutic persuasiveness, and therapeutic alliance. Phase 1 providers and survivors viewed CHAT as acceptable, easy to understand, and helpful. Participants reported that the intervention could facilitate higher engagement in this population as the web-based modality is anonymous, easily accessible, and brief. Participants offered helpful suggestions for improving CHAT by updating images, increasing content personalization, reducing text, and making users aware that the intervention is confidential. The recommendations of phase 1 participants were incorporated into CHAT. Phase 2 survivors viewed the revised intervention and found it highly acceptable (mean 4.1 out of 5, SD 1.29). A total of 4 themes encapsulated participant's favorite aspects of CHAT: (1) content and features, (2) accessible and easy to use, (3) education, and (4) personalization. Six survivors denied disliking any aspect. The themes on recommended changes included content and features, brevity, personalization, and language access. Participants provided dissemination recommendations.

CONCLUSIONS

Overall, CHAT was acceptable among victim service professionals and survivors. Positive reactions to CHAT show promise for future research investigating the efficacy and potential benefit of CHAT when integrated into services for people with traumatic stress and alcohol misuse after recent IPV and SA.

If we build it, will they come? A scoping review of objective engagement metrics in asynchronous psychosocial telehealth interventions for breast cancer survivors

Burgeoning technologies and the COVID-19 pandemic resulted in a boom of telehealth for immunocompromised patients, such as those with cancer. Telehealth modalities overcome barriers and promote accessibility to care. Currently, efficacious psychosocial interventions exist to address negative aftereffects of a cancer diagnosis and treatment. Many of these interventions often incorporate asynchronous telehealth (e.g., web-based, smartphone mobile app) features. However, asynchronous platforms are limited by suboptimal engagement. Subjective indicators of perceived engagement in the forms of acceptability, feasibility, and adherence are often captured, yet prior research has found discrepancies between perceived and actual engagement. The FITT (frequency, intensity, time/duration, type of engagement) model, originally developed for use to quantify engagement within exercise trials, provides a framework to assess objective engagement of psychosocial interventions for breast cancer. Using 14 keywords and searching six databases through 11/2023, 56 studies that used asynchronous telehealth interventions in breast cancer were identified. All FITT domains were reported at least once across studies with intensity metrics most commonly reported. Nine metrics were described across FITT domains. Human-centered design principles to guide telehealth development and privacy considerations are discussed. Findings offer suggestions for how to represent and optimize objective engagement in asynchronous telehealth cancer care.

Design of mobile and website health application devices for drug tolerability in hereditary fructose intolerance

BACKGROUND

Hereditary fructose intolerance (HFI) is a rare metabolic disease caused by aldolase B deficiency. The aim of our study was to analyse excipient tolerability in patients with HFI and other related diseases and to design mobile and website health applications to facilitate the search for drugs according to their tolerance.

RESULTS

A total of 555 excipients listed in the Spanish Medicines Agency database (July 2023) were classified as suitable for HFI patients, suitable with considerations ((glucose and glucose syrup, intravenous sucrose, oral mannitol, polydextrose, gums and carrageenans, ethanol, sulfite caramel and vanilla), not recommended (intravenous mannitol) and contraindicated (fructose, oral sucrose, invert sugar, sorbitol, maltitol, lactitol, isomaltitol, fruit syrups, honey, sucrose esters and sorbitol esters). Glucose and glucose syrup were classified as suitable with considerations due to its possible fructose content and their potential endogenous fructose production. For other related intolerances, wheat starch was contraindicated and oatmeal was not recommended in celiac disease; oral lactose and lactose-based coprocessed excipient (Cellactose®) were not recommended in lactose intolerance; and glucose, invert sugar and oral sucrose were not recommended in diabetes mellitus. The applications were named IntoMed®. Results are listed in order of tolerability (suitable drugs appear first and contraindicated drugs at the end), and they are accompanied by a note detailing their classified excipients. If a drug contains excipients within different categories, the overall classification will be the most restrictive. The apps are also able to classify substances with the same criteria if they act as active ingredients. The tools exhibited good usability (82.07 ± 13.46 points on the System Usability Scale [range: 0-100]) on a sample of HFI patients, their families and health care professionals.

CONCLUSIONS

IntoMed® is a tool for finding information about the tolerability of drugs according to excipients for patients with HFI and other related intolerances, with good usability. It is a fast and reliable system that covers the current excipient legislation and expands on it with other specific information: HFI patients should be alert for excipients such as mannitol (especially in intravenous drugs), fruit syrups, honey, sulfite caramel or vanilla. Glucose might contain or produce fructose, and special precaution is needed because of potential errors in their composition.

Benchmarking usability of patient portals in Estonia, Finland, Norway, and Sweden

INTRODUCTION

Poor usability is a barrier to widespread adoption of electronic health records (EHR). Providing good usability is especially challenging in the health care context, as there is a wide variety of patient users. Usability benchmarking is an approach for improving usability by evaluating and comparing the strength and weaknesses of systems. The main purpose of this study is to benchmark usability of patient portals across countries.

METHODS

A mixed-methods survey approach was applied to benchmark the national patient portals offering patient access to EHR in Estonia, Finland, Norway, and Sweden. These Nordic countries have similar public healthcare systems, and they are pioneers in offering patients access to EHR for several years. In a survey of 29,334 patients, both patients' quantitative ratings of usability and their qualitative descriptions of very positive and very negative peak experiences of portal use were collected.

RESULTS

The usability scores ranged from good to fair level of usability. The narratives of very positive and very negative experiences included the benefits of the patient portals and experienced usability issues. The regression analysis of results showed that very positive and negative experiences of patient portal use explain 19-35% of the variation of usability scores in the four countries. The percentage of patients who reported very positive or very negative experiences in each country was unrelated to the usability scores across countries.

CONCLUSIONS

The survey approach could be used to evaluate usability with a wide variety of users and it supported learning from comparison across the countries. The combination of quantitative and qualitative data provided an approximation of the level of the perceived usability, and identified usability issues to be improved and useful features that patients appreciate. Further work is needed to improve the comparability of the varied samples across countries.

User-centered development of an mHealth app for cardiovascular prevention

Background

Mobile health (mHealth) apps can be used for cardiovascular disease (CVD) prevention. User-centered design, evidence-based content and user testing can be applied to ensure a high level of usability and adequate app access.

Objective

To develop and evaluate an mHealth app (HerzFit) for CVD prevention.

Methods

HerzFit´s development included a user-centered design approach and guideline-based content creation based on the identified requirements of the target group. Beta testing and a preliminary usability evaluation of the HerzFit prototype were performed. For evaluation, German versions of the System Usability Scale (SUS) and the mHealth App Usability Questionnaire (GER-MAUQ) as well as free text feedback were applied.

Results

User-centered design thinking led to the definition of four personas. Based on their requirements, HerzFit enables users to individually assess, monitor, and optimize their cardiovascular risk profile. Users are also provided with a variety of evidence-based information on CVD and their risk factors. The user interface and system design followed the identified functional requirements. Beta-testers provided feedback on the structure and functionality and rated the usability of HerzFit´s prototype as slightly above average both in SUS and GER-MAUQ rating. Participants positively noted the variety of functions and information presented in HerzFit, while negative feedback mostly concerned wearable synchronization.

Conclusions

The present study demonstrates the user-centered development of a guideline-based mHealth app for CVD prevention. Beta-testing and a preliminary usability study were used to further improve the HerzFit app until its official release.

Development of a patient-centered transition program for stroke survivors and their informal caregivers, combining case-management and access to an online information platform: A user-centered design approach

Background

During the hospital-to-home transition, stroke survivors and their caregivers face a significant lack of support and information which impacts their psychosocial recovery. We aimed to co-design a program combining individual support by a trained case-manager (dedicated professional providing individual support) and an online information platform to address needs of stroke survivors and caregivers.

Methods

A two-step methodology was used. The first step followed a "user-centered design" approach during four workshops with stroke survivors, caregivers, and healthcare professionals to develop the platform and define the case-manager profile. The second step was a usability test of the platform following a Think Aloud method with patients and caregivers. The workshops and interviews were analyzed following a qualitative thematic analysis. The analysis of Think Aloud interviews was based on User Experience Honeycomb framework by Morville.

Results

Eight participants attended the workshops: two patients, two caregivers, three nurses, and a general practitioner. Activities, training, and skills of the case-manager were defined according to stroke survivors and caregivers needs. Name, graphics, navigation, and content of the platform were developed with the participants, a developer and a graphic designer. The usability of the platform was tested with 5 patients and 5 caregivers. The Think Aloud confirmed satisfaction with graphics and content but a need for improvement regarding the navigability. An update of the platform was conducted in order to answer the needs expressed by participants.

Conclusion

We developed, with a participatory approach, a patient-centered transition program, which will be evaluated in a randomized controlled trial.

Toward Personalized Orthopedic Care: Validation of a Smart Knee Brace

Introduction

Wearable technology offers a promising solution to advance current rehabilitation strategies for post-operative orthopedic care. The aim of this study was to determine the level of agreement and concurrent validity of a smart knee brace compared to the gold standard measurement system GAITRite® for assessing lower limb gait parameters.

Methods

Thirty-four healthy participants were fitted with the smart knee brace (Digital Knee®) on their dominant limb. Gait parameters (stride length, stride time, and gait velocity) were measured simultaneously using the Digital Knee® and the GAITRite® electronic walkway. Two walks were performed at a comfortable speed and two at a fast-walking speed.

Results

At a comfortable walking speed, stride time was moderately valid (ICC2,1 = 0.66 s), and stride length and gait velocity demonstrated poor validity (ICC2,1 = 0.29; ICC2,1 = 0.41). All gait parameters demonstrated poor validity at a fast-walking speed (ICC2,1 = -0.16 to -0.01). Bias ranged from -0.08 to 0.28, with more clinically acceptable percentage errors at a comfortable walking speed (14.1-30%) versus at a fast-walking speed (26.4-42.6%). Gait velocity and stride length had substantially higher biases in the fast-walking speed compared to the comfortable walking speed (0.28 ± 0.39 m s-1 vs. 0.02 ± 0.21 m s-1; 0.15 ± 0.23 m vs. -0.04 ± 0.17 m). Limits of agreement were considered narrower for stride time compared to stride length and gait velocity.

Conclusion

The Digital Knee® is a promising approach to improving post-operative rehabilitation outcomes in patients with osteoarthritis. The Digital Knee® demonstrated good agreement and moderate concurrent validity for measuring gait metrics at a comfortable walking speed. These findings highlight the opportunity of the wearable sensor as an intervention for post-operative orthopedic care. This was a laboratory-based study; thus, further research is required to validate the wearable sensor in real-world contexts and in patients with knee pathologies. Further, refinement of the algorithm for measuring gait metrics at slow- and fast-walking speed with the Digital Knee® is warranted.

Effectiveness of self-reported management program of cancer patients

Objective

This study aimed to analyze the effect of Smart Cancer Care program on the quality of life and ease of chemotherapy continuation in cancer patients and the effect of additional tele-management on frequency of use and satisfaction with the Smart Cancer Care program.

Methods

'Smart Cancer Care' is a mobile program that allows cancer patients undergoing chemotherapy to report symptoms of adverse events and receive remote management. In this study, patients were randomly assigned to three groups: Group A, who received only classical face-to-face management; Group B, who used the Smart Cancer Care program as addition; and Group C, who used the Smart Cancer Care program and received telephone management. After 12 weeks of follow-up, the effectiveness of using the Smart Cancer Care program was analyzed by examining the quality of life, ease of maintaining chemotherapy, and unplanned hospital visits in each group. The frequency of use and satisfaction with the Smart Cancer Care program were also analyzed.

Results

Cancer patients who used the Smart Cancer Care program had 1.93-fold (1.15-3.25) higher overall quality of life than those who did not. This became 2.33-fold (1.34-4.04) higher when phone care was added. Patients with tele-management were significantly more likely to use the Smart Cancer Care program (odds ratio (OR) = 25.80; 95% confidence interval (CI), 11.28-58.97).

Conclusions

A mobile self-reported management program has a positive effect on the quality of life of cancer patients undergoing chemotherapy. Tele-management is conducive to active and effective use of this program.

A web-based resource for exercise training in children treated for brain tumours to improve cognitive sequelae: Development and usability

Improving cognitive sequelae in children treated for brain tumours (CTBT) requires accessible interventions. While instructor-led exercise in a hospital setting is efficacious, it is not extended to communities.

Objectives

We aimed to (i) develop a website with educational resources/tools for community health and fitness professionals (HFP) to deliver exercise for CTBT in community settings to improve cognition and (ii) assess its usability by community HFP. It was hypothesized that the website would be learnable, clear, satisfactory and efficient to deliver exercise.

Methods

A scoping review determined the state of eHealth resources to support exercise for CTBT and identified knowledge and resource gaps. Three focus groups with HFP who served cancer survivors in hospital or community settings (n = 13) identified user needs; content analysis identified themes. Gaps from the scoping review and themes from focus groups informed website content. A questionnaire assessed its usability by community HFP (n = 4). Descriptive statistics inferred the website's learnability, clarity, satisfaction and efficiency. Open-ended responses identified issues.

Results

The scoping review revealed a lack of eHealth resources supporting exercise to improve cognition in CTBT and education for HFP to deliver exercise. Six themes were identified in the focus groups. HFP rated the website as sufficiently learnable, clear, satisfactory and efficient. Two minor issues were reported and addressed.

Conclusion

The website marks one of the first eHealth resources to increase accessibility of intervention to improve cognitive sequelae and ultimately quality of life in CTBT. HFP also gain access to education and tools to deliver exercise in community settings.

Development of a Competency-Based Triage Education Application and Usability Testing for Triage Training Based on the Korean Triage and Acuity Scale

BACKGROUND

Improving the quality of emergency nursing care requires enhancing triage competency through education programs that integrate the content of the Korean Triage and Acuity Scale, a triage system specifically developed for the Korean context. Thus, this study developed a competency-based triage education application (CTEA), based on the Korean Triage and Acuity Scale, and evaluated its effectiveness through usability testing.

METHOD

The CTEA used various teaching methods and game mechanisms, including lectures, case studies, and video-based scenarios, to improve triage competency.

RESULTS

In the usability testing phase, the CTEA was refined through expert heuristic evaluation and user interviews conducted with the think-aloud method. Six themes were derived from the interviews, highlighting the need for a well-structured program with realistic scenarios, easy accessibility, gamification, functional improvements, and future triage educational applications.

CONCLUSION

These findings suggest that the CTEA is effective and satisfactory for users and can contribute to improving the triage competency of emergency nurses. [J Contin Educ Nurs. 2024;55(1):33-41.].

The German version of the mHealth App Usability Questionnaire (GER-MAUQ): Translation and validation study in patients with cardiovascular disease

Objective

In Germany, only a few standardized evaluation tools for assessing the usability of mobile Health apps exist so far. This study aimed to translate and validate the English patient version for standalone apps of the mHealth App Usability Questionnaire (MAUQ) into a German version.

Methods

Following scientific guidelines for translation and cross-cultural adaptation, the patient version for standalone apps was forward and back-translated from English into German by an expert panel. In total, 53 participants who were recruited as part of the beta testing process of the recently developed mHealth app HerzFit, answered the questions of the German version of the MAUQ (GER-MAUQ) and the System Usability Scale. Subsequently, a descriptive as well as a psychometric analysis was performed to test validity and reliability.

Results

After conducting three cognitive interviews, five items were modified. The values for Cronbach alpha for the entire questionnaire and the three subscales (0.966, 0.814, 0.910, and 0.909) indicate strong internal consistency. The correlation analysis revealed that the scores of the GER-MAUQ, the subscales and the SUS were strongly correlated with each other. The correlation coefficient of the SUS and the GER-MAUQ overall score was r = 0.854, P < 0.001 and the coefficients of the subscales and the SUS were r = 0.642, P < 0.001; r = 0.866, P < 0.001 and r = 0.643, P < 0.001.

Conclusions

We have developed a novel German version of the MAUQ and demonstrated it as a reliable and valid measurement tool for assessing the usability of standalone mHealth apps from the patients' perspective. The GER-MAUQ allows a new form of standardized assessment of usability of mHealth apps for patients with cardiovascular disease in Germany. Further research with a larger sample and other samples is recommended.

Development and Formative Evaluation of a Virtual Exercise Platform for a Community Fitness Center Serving Individuals With Physical Disabilities: Mixed Methods Study

BACKGROUND

People with disabilities experience numerous barriers to being physically active, such as transportation issues, a lack of trained exercise professionals who understand disabilities, and facility access. The use of a virtual exercise platform (VEP) may provide an alternative and limit certain barriers.

OBJECTIVE

The aim of this mixed method study was to evaluate user interaction (effectiveness, efficiency, and satisfaction), the strengths and weaknesses of the user interface, and the user experience with a VEP.

METHODS

Participants were recruited from a community fitness facility that offers programs for people with disabilities. Inclusion criteria were being older than 18 years, fluent in English, and availability of internet access. Features of the VEP included articles, prerecorded videos, live Zoom classes, web-based class registration, weekly progress tracking, incentives, and surveys. A one-on-one Zoom session was scheduled with each participant, during which they completed certain tasks: (1) create an account or login, (2) register for class, (3) join class, (4) add to calendar, and (5) complete surveys. As participants completed tasks, quantitative observations (time on task, task success, rate of task completion, and number of errors by users, which determined task difficulty), qualitative observations were made and interviews were conducted at the end of the session. The "concurrent think-aloud" method was encouraged by the moderator to gauge participants' thoughts as they worked through testing. Participants also completed the System Usability Scale (SUS) and Questionnaire for User Interface Satisfaction (QUIS).

RESULTS

A total of 5 people with disabilities (3 male, 2 female), aged 36-78 (mean 54) years, with education levels from high school to PhD, were recruited. Devices used for testing included a laptop (n=3), a Chromebook (n=1), and a desktop (n=1). All participants completed tasks #1 and #2 without errors but could not complete task #4. One participant completed task #5 with difficulty and another completed task #3 with difficulty. The average time to complete each task was: (1) 82 seconds (55-110), (2) 11 seconds (4-21), (3) 9 seconds (5-27), and (4) 921.5 seconds (840-958). The mean SUS score was 86.5/100, and the mean user QUIS score was 8.08 out of 10. Qualitative observations indicated that the system was simple, user-friendly, and accessible.

CONCLUSIONS

People with disabilities reported high usability and user satisfaction with the web-based exercise platform, and the system appears to be an efficient and effective physical activity option.

An Online Psychological Program for Adolescents and Young Adults With Headaches: Iterative Design and Rapid Usability Testing

BACKGROUND

Headache disorders are common, debilitating health problems. Cognitive-behavioral therapy (CBT) is recommended but rarely easily available. With the use of the internet and communication technologies among youth and young adults, these individuals could be self-trained in CBT skills. There is an increasing number of internet-based interventions for headaches, but there has been little research into the usability of these interventions because evaluating usability across the intervention development life cycle is costly. We developed an internet-based CBT program, the Specialized Program for Headache Reduction (SPHERE). While developing it, we aimed to improve SPHERE through rapid usability testing cycles.

OBJECTIVE

This study aims to presents a rapid and affordable usability testing approach that can be performed throughout the intervention development life cycle. This paper also provides evidence of the usability of SPHERE.

METHODS

We used the "think aloud" usability testing method based on Krug's approach to test user interaction within a lab setting. This was followed by a short posttest interview. We planned to test SPHERE with 3-5 participants testing the same part of the program each cycle. Both the design and development team and the research team actively participated in the usability testing process. Observers independently identified the top 3 usability issues, rated their severity, and conducted debriefing sessions to come to consensus on major issues and generate potential solutions.

RESULTS

The testing process allowed major usability issues to be identified and rectified rapidly before piloting SPHERE in a real-world context. A total of 2 cycles of testing were conducted. Of the usability issues encountered in cycles 1 and 2, a total of 68% (17/25) and 32% (12/38), respectively, were rated as major, discussed, and fixed.

CONCLUSIONS

This study shows that rapid usability testing is an essential part of the design process that improves program functionality and can be easy and inexpensive to undertake.

Exploring usability characteristics in computer-based digital health technologies for family caregivers of people with chronic progressive conditions: a scoping review protocol

OBJECTIVE

The objective of this scoping review is to map the literature on how usability is considered during the design and/or evaluation of computer-based digital health technologies for family caregivers of persons with chronic progressive conditions.

INTRODUCTION

Computer-based digital health technologies offer convenient alternatives for delivering interventions to caregivers of people with chronic progressive conditions. Usability is a critical component of good practice in developing and implementing health and social care technologies; however, we need to determine whether usability is incorporated in the design and/or evaluation of computer-based digital health technologies for caregivers of people with chronic progressive conditions. Within this context, a broad overview of the existing literature on usability in computer-based digital health technologies is needed.

INCLUSION CRITERIA

We will include studies published from 2012 to the present that describe usability characteristics of computer-based digital health technologies targeting adult (≥18 years old) family caregivers of people with chronic progressive conditions, regardless of study design or setting.

METHODS

We will use the JBI methodology for scoping reviews. We will conduct searches of MEDLINE (Ovid), PsycINFO (Ovid), CINAHL (EBSCOhost), and Web of Science Core Collection to capture eligible studies. After the results are deduplicated, 2 independent reviewers will assess each study for eligibility and extract data from the included studies. Conflicts will be resolved through discussion or with a third reviewer. Data analysis will use a textual narrative synthesis approach.

REVIEW REGISTRATION

Open Science Framework osf.io/w4vk5.