Conceptualizing Usability for the eHealth Context: Content Analysis of Usability Problems of eHealth Applications

Afraid of the dentist? There's an app for that: Development and usability testing of a cognitive behavior therapy-based mobile app

OBJECTIVES

Although several brief cognitive behavior therapy (CBT)-based treatments for dental fear have proven efficacious, these interventions remain largely unavailable outside of the specialty clinics in which they were developed. Leveraging technology, we sought to increase access to treatment for individuals with dental fear through the development of a mobile application (Dental FearLess).

MATERIALS AND METHODS

To assess the resonance of our app as an avenue for dental fear treatment, we conducted a study assessing the usability, feasibility, and acceptability of the beta app. Participants with moderate to severe dental fear (N = 80) completed the app and reported on the perceived usability of the mobile interface (Systems Usability Scale, SUS; α = .82) and credibility of the intervention (Credibility and Expectancy Questionnaire, CEQ; α = .88). A sub-sample of participants naïve to the app (n = 10) completed the app during a think-aloud procedure, sharing their candid thoughts and reactions while using the app, prior to reporting on usability and credibility metrics.

RESULTS

Overall usability (M = 78.5, SD = 17.7) and credibility (M = 21.7, SD = 5.5) of the beta version of the app were good. The think-aloud data further corroborated the app's acceptability, while highlighting several areas for user improvement (i.e., aesthetics, navigation, engagement).

CONCLUSIONS

Usability and acceptability results are promising for the viability of an accessible, feasible, self-administered intervention for dental fear. Refinements made based on user feedback have produced a clinical-trial-ready mobile application. App refinement decisions, informed by user feedback, are representative of the larger literature-that is, of the ubiquitous negotiations m-health developers must make across treatment fidelity, usability, and engagement. Implications for future research are discussed.

A review of incidents related to health information technology in Swedish healthcare to characterise system issues as a basis for improvement in clinical practice

This study examined health information technology-related incidents to characterise system issues as a basis for improvement in Swedish clinical practice. Incident reports were collected through interviews together with retrospectively collected incidents from voluntary incident databases, which were analysed using deductive and inductive approaches. Most themes pertained to system issues, such as functionality, design, and integration. Identified system issues were dominated by technical factors (74%), while human factors accounted for 26%. Over half of the incidents (55%) impacted on staff or the organisation, and the rest on patients - patient inconvenience (25%) and patient harm (20%). The findings indicate that it is vital to choose and commission suitable systems, design out "error-prone" features, ensure contingency plans are in place, implement clinical decision-support systems, and respond to incidents on time. Such strategies would improve the health information technology systems and Swedish clinical practice.

Testing Two Online Symptom Checkers With Vulnerable Groups: Usability Study to Improve Cognitive Accessibility of eHealth Services

BACKGROUND

The popularity of eHealth services has surged significantly, underscoring the importance of ensuring their usability and accessibility for users with diverse needs, characteristics, and capabilities. These services can pose cognitive demands, especially for individuals who are unwell, fatigued, or experiencing distress. Additionally, numerous potentially vulnerable groups, including older adults, are susceptible to digital exclusion and may encounter cognitive limitations related to perception, attention, memory, and language comprehension. Regrettably, many studies overlook the preferences and needs of user groups likely to encounter challenges associated with these cognitive aspects.

OBJECTIVE

This study primarily aims to gain a deeper understanding of cognitive accessibility in the practical context of eHealth services. Additionally, we aimed to identify the specific challenges that vulnerable groups encounter when using eHealth services and determine key considerations for testing these services with such groups.

METHODS

As a case study of eHealth services, we conducted qualitative usability testing on 2 online symptom checkers used in Finnish public primary care. A total of 13 participants from 3 distinct groups participated in the study: older adults, individuals with mild intellectual disabilities, and nonnative Finnish speakers. The primary research methods used were the thinking-aloud method, questionnaires, and semistructured interviews.

RESULTS

We found that potentially vulnerable groups encountered numerous issues with the tested services, with similar problems observed across all 3 groups. Specifically, clarity and the use of terminology posed significant challenges. The services overwhelmed users with excessive information and choices, while the terminology consisted of numerous complex medical terms that were difficult to understand. When conducting tests with vulnerable groups, it is crucial to carefully plan the sessions to avoid being overly lengthy, as these users often require more time to complete tasks. Additionally, testing with vulnerable groups proved to be quite efficient, with results likely to benefit a wider audience as well.

CONCLUSIONS

Based on the findings of this study, it is evident that older adults, individuals with mild intellectual disability, and nonnative speakers may encounter cognitive challenges when using eHealth services, which can impede or slow down their use and make the services more difficult to navigate. In the worst-case scenario, these challenges may lead to errors in using the services. We recommend expanding the scope of testing to include a broader range of eHealth services with vulnerable groups, incorporating users with diverse characteristics and capabilities who are likely to encounter difficulties in cognitive accessibility.

Benchmarking usability of patient portals in Estonia, Finland, Norway, and Sweden

INTRODUCTION

Poor usability is a barrier to widespread adoption of electronic health records (EHR). Providing good usability is especially challenging in the health care context, as there is a wide variety of patient users. Usability benchmarking is an approach for improving usability by evaluating and comparing the strength and weaknesses of systems. The main purpose of this study is to benchmark usability of patient portals across countries.

METHODS

A mixed-methods survey approach was applied to benchmark the national patient portals offering patient access to EHR in Estonia, Finland, Norway, and Sweden. These Nordic countries have similar public healthcare systems, and they are pioneers in offering patients access to EHR for several years. In a survey of 29,334 patients, both patients' quantitative ratings of usability and their qualitative descriptions of very positive and very negative peak experiences of portal use were collected.

RESULTS

The usability scores ranged from good to fair level of usability. The narratives of very positive and very negative experiences included the benefits of the patient portals and experienced usability issues. The regression analysis of results showed that very positive and negative experiences of patient portal use explain 19-35% of the variation of usability scores in the four countries. The percentage of patients who reported very positive or very negative experiences in each country was unrelated to the usability scores across countries.

CONCLUSIONS

The survey approach could be used to evaluate usability with a wide variety of users and it supported learning from comparison across the countries. The combination of quantitative and qualitative data provided an approximation of the level of the perceived usability, and identified usability issues to be improved and useful features that patients appreciate. Further work is needed to improve the comparability of the varied samples across countries.

Failures and fallacies of eHealth initiatives: Are we finally able to overcome the underlying theoretical and practical orthodoxies?

The growing and ubiquitous digitalization trends embodied in eHealth initiatives have led to the widespread adoption of digital solutions in the healthcare sector. These initiatives have been heralded as a potent transformative force aiming to improve healthcare delivery, enhance patient outcomes and increase the efficiency of healthcare systems. However, despite the significant potential and possibilities offered by eHealth initiatives, the article highlights the importance of critically examining their implications and cautions against the misconception that technology alone can solve complex public health concerns and healthcare challenges. It emphasizes the need to critically consider the sociocultural context, education and training, organizational and institutional aspects, regulatory frameworks, user involvement and other important factors when implementing eHealth initiatives. Disregarding these crucial elements can render eHealth initiatives inefficient or even counterproductive. In view of that, the article identifies failures and fallacies that can hinder the success of eHealth initiatives and highlights areas where they often fall short of meeting rising and unjustified expectations. To address these challenges, the article recommends a more realistic and evidence-based approach to planning and implementing eHealth initiatives. It calls for consistent research agendas, appropriate evaluation methodologies and strategic orientations within eHealth initiatives. By adopting this approach, eHealth initiatives can contribute to the achievement of societal goals and the realization of the key health priorities and development imperatives of healthcare systems on a global scale.

Patient Portals as Facilitators of Engagement in Patients With Diabetes and Chronic Heart Disease: Scoping Review of Usage and Usability

BACKGROUND

Patient portals have the potential to improve care for chronically ill patients by engaging them in their treatment. These platforms can work, for example, as a standalone self-management intervention or a tethered link to treatment providers in routine care. Many different types of portals are available for different patient groups, providing various features.

OBJECTIVE

This scoping review aims to summarize the current literature on patient portals for patients with diabetes mellitus and chronic heart disease regarding usage behavior and usability.

METHODS

We conducted this review according to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) statement for scoping reviews. We performed database searches using PubMed, PsycInfo, and CINAHL, as well as additional searches in reviews and reference lists. We restricted our search to 2010. Qualitative and quantitative studies, and studies using both approaches that analyzed usage behavior or usability of patient portals were eligible. We mapped portal features according to broad thematic categories and summarized the results of the included studies separately according to outcome and research design.

RESULTS

After screening, we finally included 85 studies. Most studies were about patients with diabetes, included patients younger than 65 years, and were conducted in the United States. Portal features were categorized into educational/general information, reminder, monitoring, interactivity, personal health information, electronic/personal health record, and communication. Portals mostly provided educational, monitoring, and communication-related features. Studies reported on usage behavior including associated variables, usability dimensions, and suggestions for improvement. Various ways of reporting usage frequency were identified. A noticeable decline in portal usage over time was reported frequently. Age was most frequently studied in association with portal use, followed by gender, education, and eHealth literacy. Younger age and higher education were often associated with higher portal use. In two-thirds of studies reporting on portal usability, the portals were rated as user friendly and comprehensible, although measurement and reporting were heterogeneous. Portals were considered helpful for self-management through positive influences on motivation, health awareness, and behavioral changes. Helpful features for self-management were educational/general information and monitoring. Barriers to portal use were general (eg, aspects of design or general usability), related to specific situations during portal use (eg, login procedure), or not portal specific (eg, user skills and preferences). Frequent themes were aspects of design, usability, and technology. Suggestions for improvement were mainly related to technical issues and need for support.

CONCLUSIONS

The current state of research emphasizes the importance of involving patients in the development and evaluation of patient portals. The consideration of various research designs in a scoping review is helpful for a deeper understanding of usage behavior and usability. Future research should focus on the role of disease burden, and usage behavior and usability among older patients.

Towards multimodal boosting of motivation for fall-preventive physical activity in seniors: An iterative development evaluation study

Background

Many seniors need to increase their physical activity (PA) and participation in fall prevention exercise. Therefore, digital systems have been developed to support fall-preventive PA. Most of them lack video coaching and PA monitoring, two functionalities that may be relevant for increasing PA.

Objective

To develop a prototype of a system to support seniors' fall-preventive PA, which includes also video coaching and PA monitoring, and to evaluate its feasibility and user experience.

Methods

A system prototype was conceived by integrating applications for step-monitoring, behavioural change support, personal calendar, video-coaching and a cloud service for data management and co-ordination. Its feasibility and user experience were evaluated in three consecutive test periods combined with technical development. In total, 11 seniors tested the system at home for four weeks with video coaching from health care professionals.

Results

Initially, the system's feasibility was non-satisfactory due to insufficient stability and usability. However, most problems could be addressed and amended. In the third (last) test period, both seniors and coaches experienced the system prototype to be fun, flexible and awareness-raising. Interestingly, the video coaching which made the system unique compared to similar systems was highly appreciated. Nonetheless, even the users in the last test period highlighted issues due to insufficient usability, stability and flexibility. Further improvements in these areas are needed.

Conclusions

Video coaching in fall-preventive PA can be valuable for both seniors and health care professionals. High reliability, usability and flexibility of systems supporting seniors are essential.

Usability of a continuous oxygen saturation device for home telemonitoring

Background

The emergency for the COVID-19 pandemic has led to greater use of home telemonitoring devices. The aim of this study was to assess the usability of continuous home-monitoring care with an oxygen saturation device on post-COVID-19 patients.

Method

The system consists of a digital continuous pulse oximeter and a smartphone with an App, which were provided to patients. A survey composed of a standard Post-Study System Usability Questionnaire, and a satisfaction questionnaire was exploited to conduct a usability and feasibility analysis of the service.

Results

A total of 29 patients (17.2% female) with a mean age of 65 ± 11.5 years were enrolled: 20 patients were smartphone users (69%) with a mean age of 60.2 ± 9.5 years, and 9 patients (31%) did not own a smartphone (mean age 76.8 ± 5.9). The monitoring period was 1 month: a total of 444 recordings were conducted, 15 recordings per patient averagely. In total, 82% of the recordings performed did not require any intervention, while 18% led to the production of a report and subsequent intervention by a nurse who verified, together with the specialist, the need to intervene (i.e. the patient accessed the clinic for medical control and/or modification of oxygen therapy). A total of 17 patients compiled a usability questionnaire. The service was perceived as useful and well-structured, although it often required caregiver support.

Conclusions

Using continuous home-monitoring care with an oxygen saturation device seems feasible and useful for patients who could be followed at home avoiding going back to the hospital every time a trend oximetry is needed. Further improvements in connections, data flow processes, and simplifications, based on patients' feedback, are needed to scale up the service.

Patterns of acceptance and use of digital health services among the persistent frequent attenders of outpatient care: A qualitatively driven multimethod analysis

Objective

Utilising digital health services in the treatment of patients who frequently attend outpatient care could be beneficial for patients' health and the sustainability of health systems but carries the risk of digital exclusion. This study aimed to explore the patterns of acceptance and use of digital health services among frequent attenders (FAs), which may help in the assessment of patients' digital suitability.

Methods

Persistent FAs (N = 30) were recruited by random sampling from one Finnish municipality. The semistructured interviews were conducted in February-May 2021. We analysed the data with qualitative content analysis using the Unified Theory of Acceptance and Use of Technology (UTAUT) model. Additionally, we quantified the data for two-step cluster analyses to create separate cluster models that grouped FAs based on acceptance and use of (a) digital services for self-management of health and (b) telemedicine services.

Results

Based on digital self-management, FAs were defined as Self-Managers, Supported Self-Managers, and Non-Self-Managers. Based on telemedicine use, they were grouped into Telemedicine Users, Doubtful Telemedicine Users, and Telemedicine Refusers. The clusters described different opportunities, awareness, and interest in using digital health services. Referral from professionals seemed to promote digital service use. For some, digital services were not accessible.

Conclusions

Our findings emphasise the importance of assessing the suitability of FAs to digital health services, as their readiness to use may vary. Professionals should recommend digital services that support individual health to suitable patients. More accessible digital services could promote digital suitability despite functional limitations.

Systematic mapping of digital health apps - A methodological proposal based on the World Health Organization classification of interventions

Coronavirus disease 2019 was identified as a pandemic and Brazil is one of the major epicentres. One of the Brazilian states affected is Ceará, where this research group works. This group was challenged by a Hospital stakeholder to develop a communication channel with the health professionals and the coronavirus disease 2019 patient's family. This article presents a part of this whole project. The main methodological approach was the user-centred design based on user experience elements. Benchmarking was applied to understand the state-of-art of Brazilian apps that were related to coronavirus disease 2019. The research process was based on a systematic approach that was carried out by a multidisciplinary team that worked through four work cycles (identification, classification, screening, analysis). This work was based on two main points: (a) World Health Organization digital health guidelines, specifically digital health interventions (b) System Usability Scale. As a result, apps features were gathered according to the digital health interventions and their experiences were analysed on System Usability Scale. This work has provided an overview of apps that were available and how they support the coronavirus disease 2019 context. Another valuable contribution is the understanding of how the industry was satisfying the user's needs. These two results can provide a holistic view for future product development that can be used in different contexts of health issues. One of the highlighted conclusions was that digital health interventions should be adapted to the local context because these World Health Organization guidelines were open. Moreover, the System Usability Scale is an effective method to compare different digital health solutions.

Chinese Version of the mHealth App Usability Questionnaire: Cross-Cultural Adaptation and Validation

Background

Mobile health (mHealth) apps have shown the advantages of improving medication compliance, saving time required for diagnosis and treatment, reducing medical expenses, etc. The World Health Organization (WHO) has recommended that mHealth apps should be evaluated prior to their implementation to ensure their accuracy in data analysis.

Objective

This study aimed to translate the patient version of the interactive mHealth app usability questionnaire (MAUQ) into Chinese, and to conduct cross-cultural adaptation and reliability and validity tests.

Methods

The Brislin’s translation model was used in this study. The cross-cultural adaptation was performed according to experts’ comments and the results of prediction test. The convenience sampling method was utilized to investigate 346 patients who used the “Good Doctor” (“Good Doctor” is the most popular mHealth app in China), and the reliability and validity of the questionnaire were evaluated as well.

Results

After translation and cross-cultural adaptation, there were a total of 21 items and 3 dimensions: usability and satisfaction (8 items), system information arrangement (6 items), and efficiency (7 items). The content validity index was determined to be 0.952, indicating that the 21 items used to evaluate the usability of the Chinese version of the MAUQ were well correlated. The Cronbach’s α coefficient of the total questionnaire was 0.912, which revealed that the questionnaire had a high internal consistency. The values of test-retest reliability and split-half reliability of the Chinese version of the MAUQ were 0.869 and 0.701, respectively, representing that the questionnaire had a good stability.

Conclusion

The translated questionnaire has good reliability and validity in the context of Chinese culture, and it could be used as a usability testing tool for the patient version of interactive mHealth apps.

Improving usability benchmarking for the eHealth domain: The development of the eHealth UsaBility Benchmarking instrument (HUBBI)

Background

Currently, most usability benchmarking tools used within the eHealth domain are based on re-classifications of old usability frameworks or generic usability surveys. This makes them outdated and not well suited for the eHealth domain. Recently, a new ontology of usability factors was developed for the eHealth domain. It consists of eight categories: Basic System Performance (BSP), Task-Technology Fit (TTF), Accessibility (ACC), Interface Design (ID), Navigation & Structure (NS), Information & Terminology (IT), Guidance & Support (GS) and Satisfaction (SAT).

Objective

The goal of this study is to develop a new usability benchmarking tool for eHealth, the eHealth UsaBility Benchmarking Instrument (HUBBI), that is based on a new ontology of usability factors for eHealth.

Methods

First, a large item pool was generated containing 66 items. Then, an online usability test was conducted, using the case study of a Dutch website for general health advice. Participants had to perform three tasks on the website, after which they completed the HUBBI. Using Partial Least Squares Structural Equation Modelling (PLS-SEM), we identified the items that assess each factor best and that, together, make up the HUBBI.

Results

A total of 148 persons participated. Our selection of items resulted in a shortened version of the HUBBI, containing 18 items. The category Accessibility is not included in the final version, due to the wide range of eHealth services and their heterogeneous populations. This creates a constantly different role of Accessibility, which is a problem for a uniform benchmarking tool.,

Conclusions

The HUBBI is a new and comprehensive usability benchmarking tool for the eHealth domain. It assesses usability on seven domains (BSP, TTF, ID, NS, IT, GS, SAT) in which a score per domain is generated. This can help eHealth developers to quickly determine which areas of the eHealth system’s usability need to be optimized.