Methods of usability testing in the development of eHealth applications: A scoping review

Developing and validating high-value patient digital follow-up services: a pilot study in cardiac surgery

Background

The existing digital healthcare solutions demand a service development approach that assesses needs, experience, and outcomes, to develop high-value digital healthcare services. The objective of this study was to develop a digital transformation of the patients’ follow-up service after cardiac surgery, based on a remote patient monitoring service that would respond to the real context challenges.

Methods

The study followed the Design Science Research methodology framework and incorporated concepts from the Lean startup method to start designing a minimal viable product (MVP) from the available resources. The service was implemented in a pilot study with 29 patients in 4 iterative develop-test-learn cycles, with the engagement of developers, researchers, clinical teams, and patients.

Results

Patients reported outcomes daily for 30 days after surgery through Internet-of-Things (IoT) devices and a mobile app. The service’s evaluation considered experience, feasibility, and effectiveness. It generated high satisfaction and high adherence among users, fewer readmissions, with an average of 7 ± 4.5 clinical actions per patient, primarily due to abnormal systolic blood pressure or wound-related issues.

Conclusions

We propose a 6-step methodology to design and validate a high-value digital health care service based on collaborative learning, real-time development, iterative testing, and value assessment.

SAlBi educa (Tailored Nutrition App for Improving Dietary Habits): Initial Evaluation of Usability

In recent years, the use of applications to improve dietary habits has increased. Although numerous nutrition apps are available on the market, only few have been developed by health and nutrition professionals based on scientific evidence and subsequently tested to prove their usability. The main objective of this study was to design, develop, and evaluate the usability of a tailored nutrition application to be used to promote healthy eating habits. In order to decide app design and content, three focus groups took place with fifteen professionals from primary healthcare, nutrition, and food science and computer science, as well as expert users. For the general and feedback message design, a reference model based on the scientific literature was developed. To address the multi-perspective approach of users' and external healthcare professionals' feedback, a one-day pilot testing with potential users and healthcare professionals was conducted with four focus groups. To evaluate the relevance and potential usability of the app a 1-month pilot test was conducted in a real-life environment. A total of 42 volunteers participated in the one-day pilot testing, and 39 potential users participated in the 1-month pilot test. The SAlBi educa app developed includes an online dietary record, a self-monitoring tool to evaluate dietary patterns, general and feedback messages, and examples of traditional Mediterranean recipes. The usability study showed that volunteers think that SAlBi educa is pleasant (59%) and easy to learn to use (94%). Over 84% of the volunteers declared that the nutritional messages were clear and useful. Volunteers stated that general and tailored recommendations, as well as self-monitoring, were SAlBi educa's most motivating and useful features. SAlBi educa is an innovative, user-friendly nutritional education tool with the potential to engage and help individuals to follow dietary habits based on the Mediterranean model.

A Veteran-Centric Web-Based Decision Aid for Lung Cancer Screening: Usability Analysis

BACKGROUND

Web-based tools developed to facilitate a shared decision-making (SDM) process may facilitate the implementation of lung cancer screening (LCS), an evidence-based intervention to improve cancer outcomes. Veterans have specific risk factors and shared experiences that affect the benefits and potential harms of LCS and thus may value a veteran-centric LCS decision tool (LCSDecTool).

OBJECTIVE

This study aims to conduct usability testing of an LCSDecTool designed for veterans receiving care at a Veteran Affairs medical center.

METHODS

Usability testing of the LCSDecTool was conducted in a prototype version (phase 1) and a high-fidelity version (phase 2). A total of 18 veterans and 8 clinicians participated in phase 1, and 43 veterans participated in phase 2. Quantitative outcomes from the users included the System Usability Scale (SUS) and the End User Computing Satisfaction (EUCS) in phase 1 and the SUS, EUCS, and Patient Engagement scale in phase 2. Qualitative data were obtained from observations of user sessions and brief interviews. The results of phase 1 informed the modifications of the prototype for the high-fidelity version. Phase 2 usability testing took place in the context of a pilot hybrid type 1 effectiveness-implementation trial.

RESULTS

In the phase 1 prototype usability testing, the mean SUS score (potential range: 0-100) was 81.90 (SD 9.80), corresponding to an excellent level of usability. The mean EUCS score (potential range: 1-5) was 4.30 (SD 0.71). In the phase 2 high-fidelity usability testing, the mean SUS score was 65.76 (SD 15.23), corresponding to a good level of usability. The mean EUCS score was 3.91 (SD 0.95); and the mean Patient Engagement scale score (potential range 1 [low] to 5 [high]) was 4.62 (SD 0.67). The median time to completion in minutes was 13 (IQR 10-16). A thematic analysis of user statements documented during phase 2 high-fidelity usability testing identified the following themes: a low baseline level of awareness and knowledge about LCS increased after use of the LCSDecTool; users sought more detailed descriptions about the LCS process; the LCSDecTool was generally easy to use, but specific navigation challenges remained; some users noted difficulty understanding medical terms used in the LCSDecTool; and use of the tool evoked veterans' struggles with prior attempts at smoking cessation.

CONCLUSIONS

Our findings support the development and use of this eHealth technology in the primary care clinical setting as a way to engage veterans, inform them about a new cancer control screening test, and prepare them to participate in an SDM discussion with their provider.

From Digital Health to Digital Well-being: Systematic Scoping Review

Background

Digital health refers to the proper use of technology for improving the health and well-being of people and enhancing the care of patients through the intelligent processing of clinical and genetic data. Despite increasing interest in well-being in both health care and technology, there is no clear understanding of what constitutes well-being, which leads to uncertainty in how to create well-being through digital health. In an effort to clarify this uncertainty, Brey developed a framework to define problems in technology for well-being using the following four categories: epistemological problem, scope problem, specification problem, and aggregation problem.

Objective

This systematic scoping review aims to gain insights into how to define and address well-being in digital health.

Methods

We followed the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) checklist. Papers were identified from 6 databases and included if they addressed the design or evaluation of digital health and reported the enhancement of patient well-being as their purpose. These papers were divided into design and evaluation papers. We studied how the 4 problems in technology for well-being are considered per paper.

Results

A total of 117 studies were eligible for analysis (n=46, 39.3% design papers and n=71, 60.7% evaluation papers). For the epistemological problem, the thematic analysis resulted in various definitions of well-being, which were grouped into the following seven values: healthy body, functional me, healthy mind, happy me, social me, self-managing me, and external conditions. Design papers mostly considered well-being as healthy body and self-managing me, whereas evaluation papers considered the values of healthy mind and happy me. Users were rarely involved in defining well-being. For the scope problem, patients with chronic care needs were commonly considered as the main users. Design papers also regularly involved other users, such as caregivers and relatives. These users were often not involved in evaluation papers. For the specification problem, most design and evaluation papers focused on the provision of care support through a digital platform. Design papers used numerous design methods, whereas evaluation papers mostly considered pre-post measurements and randomized controlled trials. For the aggregation problem, value conflicts were rarely described.

Conclusions

Current practice has found pragmatic ways of circumventing or dealing with the problems of digital health for well-being. Major differences exist between the design and evaluation of digital health, particularly regarding their conceptualization of well-being and the types of users studied. In addition, we found that current methodologies for designing and evaluating digital health can be improved. For optimal digital health for well-being, multidisciplinary collaborations that move beyond the common dichotomy of design and evaluation are needed.

Usability Testing of a Digital Assessment Routing Tool for Musculoskeletal Disorders: An Iterative Convergent Mixed Methods Study (Preprint)

Background

Musculoskeletal disorders negatively affect millions of patients worldwide, placing significant demand on health care systems. Digital technologies that improve clinical outcomes and efficiency across the care pathway are development priorities. We developed the musculoskeletal Digital Assessment Routing Tool (DART) to enable self-assessment and immediate direction to the right care.

Objective

We aimed to assess and resolve all serious DART usability issues to create a positive user experience and enhance system adoption before conducting randomized controlled trials for the integration of DART into musculoskeletal management pathways.

Methods

An iterative, convergent mixed methods design was used, with 22 adult participants assessing 50 different clinical presentations over 5 testing rounds across 4 DART iterations. Participants were recruited using purposive sampling, with quotas for age, habitual internet use, and English-language ability. Quantitative data collection was defined by the constructs within the International Organization for Standardization 9241-210-2019 standard, with user satisfaction measured by the System Usability Scale. Study end points were resolution of all grade 1 and 2 usability problems and a mean System Usability Scale score of ≥80 across a minimum of 3 user group sessions.

Results

All participants (mean age 48.6, SD 15.2; range 20-77 years) completed the study. Every assessment resulted in a recommendation with no DART system errors and a mean completion time of 5.2 (SD 4.44, range 1-18) minutes. Usability problems were reduced from 12 to 0, with trust and intention to act improving during the study. The relationship between eHealth literacy and age, as explored with a scatter plot and calculation of the Pearson correlation coefficient, was performed for all participants (r=–0.2; 20/22, 91%) and repeated with a potential outlier removed (r=–0.23), with no meaningful relationships observed or found for either. The mean satisfaction for daily internet users was highest (19/22, 86%; mean 86.5, SD 4.48; 90% confidence level [CL] 1.78 or –1.78), with nonnative English speakers (6/22, 27%; mean 78.1, SD 4.60; 90% CL 3.79 or –3.79) and infrequent internet users scoring the lowest (3/22, 14%; mean 70.8, SD 5.44; 90% CL 9.17 or –9.17), although the CIs overlap. The mean score across all groups was 84.3 (SD 4.67), corresponding to an excellent system, with qualitative data from all participants confirming that DART was simple to use.

Conclusions

All serious DART usability issues were resolved, and a good level of satisfaction, trust, and willingness to act on the DART recommendation was demonstrated, thus allowing progression to randomized controlled trials that assess safety and effectiveness against usual care comparators. The iterative, convergent mixed methods design proved highly effective in fully evaluating DART from a user perspective and could provide a blueprint for other researchers of mobile health systems.

International Registered Report Identifier (IRRID)

RR2-10.2196/27205

Patients' and providers' perspectives on e-health applications designed for self-care in association with surgery - a scoping review

Background

Before and after major surgery, access to information in a user-friendly way is a prerequisite for patients to feel confident in taking on the responsibility for their surgical preparation and recovery. Several e-health applications have been developed to support patients perioperatively. The aim of this review was to give an overview of e-health applications designed for self-care associated with surgery by providing a scoping overview of perspectives from providers and patients.

Methods

We searched the following data sources to identify peer-reviewed quantitative and qualitative studies published between 2015 and 2020: CINAHL, Google Scholar, MEDLINE, PsycInfo, Web of Science, and Scopus. After identifying 960 titles, we screened 638 abstracts, of which 72 were screened in full text. Protocol register: 10.17605/OSF.IO/R3QND.

Results

We included 15 studies which met our inclusion criteria. Data from several surgical contexts revealed that the most common self-care actions in e-health applications were preoperative preparations and self-assessments of postoperative recovery. Motivational factors for self-care were information, combined with supportive reminders and messages, and chat features. Although there was great variance in research designs and technical solutions, a willingness to engage with and adhere to e-health seemed to increase patients’ self-care activities and thereby accelerate return to work and normal activities. In addition, the need for physical visits seemed to decrease. Even though age groups were not primarily studied, the included studies showed that adult patients of any age engaged in surgical self-care supported by e-health. The providers’ perspectives were not found.

Conclusions

E-health applications supporting perioperative self-care indicated a positive impact on recovery. However, experiences of healthcare professionals delivering e-health associated with surgery are missing. Additionally, studies based on patients’ perspectives regarding willingness, adherence, and motivation for self-care supported by e-health are sparse. A need for studies examining the supporting role of e-health for self-care in the surgical context is therefore needed.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-022-07718-8.

The usability, feasibility and fidelity of the Ethics Quarter e-learning intervention for nurse managers

BACKGROUND

Nurse managers (NMs) expect support to carry out their ethical activities in a complex health care environment. In this study, the Ethics Quarter (EQ) is suggested as a new educational ethics e-learning intervention for nurse managers. The aim of this study was to evaluate the usability, feasibility and fidelity of the EQ. The goal was to create a new way to support NMs' ethical activity profile (developing one's own ethics knowledge, influencing ethical issues, conducting or implementing ethics research, identifying and solving ethical problems) for the use of healthcare organizations.

METHODS

The EQ was developed under guidance of the criteria for complex interventions in health care (CReDECI2) guideline. A cross-sectional survey was conducted within the intervention group after a randomized controlled trial (the main study is registered in ClinicalTrials.gov with the identifier: 04234503). The participants were NM members of the Union of Health and Social Care Professionals in Finland (n = 95).

RESULTS

A system usability scale (SUS) assessed the overall usability of EQ as good (a mean SUS score of 85.40 out of 100). Positive feedback about the EQ's feasibility was reported in structured and open questions (a good, necessary and practical research knowledge-based e-learning intervention for all nurse managers) and recommendations for further development (intervention contents could be even more challenging and interactive) were highlighted. Fidelity, measured with Google Analytics, reported shorter time used by NMs on the EQ education than estimated.

CONCLUSIONS

The findings support the high usability, feasibility and average fidelity of the EQ intervention and its potential while also providing evidence for the development of future ethics education. Health care organizations would benefit from adopting the EQ to support the ethical activities and ethical activity profile of NMs. Additionally, this study provides an example of ethics intervention development and evaluation in nursing research.

Usability Testing of a Patient-Centered Mobile Health App for Supporting and Guiding the Pediatric Emergency Department Patient Journey: Mixed Methods Study

BACKGROUND

Patient experience in emergency departments (EDs) remains often suboptimal and can be a source of stress, particularly in pediatric settings. In an attempt to support patients and their families before, during, and after their visit to a pediatric ED, a mobile health (mHealth) app was developed by a multidisciplinary team based on patient-centered care principles.

OBJECTIVE

This study aims to evaluate the usability (effectiveness, efficiency, and satisfaction) of a new mHealth app, InfoKids, by potential end users through usability testing.

METHODS

The app was assessed through an in-laboratory, video-recorded evaluation in which participants had to execute 9 goal-oriented tasks, ranging from account creation to the reception of a diagnostic sheet at the end of the emergency care episode. Effectiveness was measured based on the task completion rate, efficiency on time on task, and user satisfaction according to answers to the System Usability Scale questionnaire. Think-aloud usability sessions were also transcribed and analyzed. Usability problems were rated for their severity and categorized according to ergonomic criteria.

RESULTS

A total of 17 parents participated in the study. The overall completion rate was 97.4% (149/153). Overall, they reported good effectiveness, with the task successfully completed in 88.2% (135/153) of cases (95% CI 83%-93%). Each task, with the exception of the first, created difficulties for some participants but did not prevent their completion by most participants. Users reported an overall good to excellent perceived usability of the app. However, ergonomic evaluation identified 14 usability problems occurring 81 time. Among these, 50% (7/14) were serious as their severity was rated as either major or catastrophic and indicated areas of improvements for the app. Following the suggested usability improvements by participants, mitigation measures were listed to further improve the app and avoid barriers to its adoption.

CONCLUSIONS

Usability of the InfoKids app was evaluated as good to excellent by users. Areas of improvement were identified, and mitigation measures were proposed to inform its development toward a universal app for all ED patients visiting a digitalized institution. Its contribution could also be useful in paving the way for further research on mobile apps aimed at supporting and accompanying patients in their care episodes, as research in this area is scarce.

Development of a Digital Support Application With Evidence-Based Content for Sustainable Return to Work for Persons With Chronic Pain and Their Employers: User-Centered Agile Design Approach

BACKGROUND

Persons with chronic pain experience a lack of support after completing rehabilitation and the responsibility for the return-to-work (RTW) process is taken over by the employer. In addition, employers describe not knowing how to support their employees. Smartphone apps have been increasingly used for self-management, but there is a lack of available eHealth apps with evidence-based content providing digital support for persons with chronic pain and their employers when they return to work.

OBJECTIVE

This study aims to describe the development of a digital support application with evidence-based content that includes a biopsychosocial perspective on chronic pain for sustainable RTW for persons with chronic pain and their employers (SWEPPE [Sustainable Worker Digital Support for Persons With Chronic Pain and Their Employers]).

METHODS

A user-centered agile design approach was applied. The multidisciplinary project team consisted of health care researchers, a user representative, and a software team. A total of 2 reference groups of 7 persons with chronic pain and 4 employers participated in the development process and usability testing. Mixed methods were used for data collection. The design was revised using feedback from the reference groups. The content of SWEPPE was developed based on existing evidence and input from the reference groups.

RESULTS

The reference groups identified the following as important characteristics to include in SWEPPE: keeping users motivated, tracking health status and work situation, and following progress. SWEPPE was developed as a smartphone app for the persons with chronic pain and as a web application for their employers. SWEPPE consists of six modules: the action plan, daily self-rating, self-monitoring graphs, the coach, the library, and shared information with the employer. The employers found the following functions in SWEPPE to be the most useful: employees' goals related to RTW, barriers to RTW, support wanted from the employer, and the ability to follow employees' progress. The persons with chronic pain found the following functions in SWEPPE to be the most useful: setting a goal related to RTW, identifying barriers and strategies, and self-monitoring. Usability testing revealed that SWEPPE was safe, useful (ie, provided relevant information), logical, and easy to use with an appealing interface.

CONCLUSIONS

This study reports the development of a digital support application for persons with chronic pain and their employers. SWEPPE fulfilled the need of support after an interdisciplinary pain rehabilitation program with useful functions such as setting a goal related to RTW, identification of barriers and strategies for RTW, self-monitoring, and sharing information between the employee and the employer. The user-centered agile design approach contributed to creating SWEPPE as a relevant and easy-to-use eHealth intervention. Further studies are needed to examine the effectiveness of SWEPPE in a clinical setting.

Nutrition-Related Mobile Apps in the French App Stores: Assessment of Functionality and Quality

BACKGROUND

The global burden of disease attributes 20% of deaths to poor nutrition. Although hundreds of nutrition-related mobile apps have been created, and these have been downloaded by millions of users, the effectiveness of these technologies on the adoption of healthy eating has had mixed.

OBJECTIVE

The aim of this study was to review which nutrition-related mobile apps are currently available on the French market and assess their quality.

METHODS

We screened apps on the Google Play Store and the French Apple App Store, from March 10 to 17, 2021, to identify those related to nutritional health. A shortlist of 15 apps was identified, and each was assessed using the French version of the Mobile App Rating Scale: 8 dietitians and nutritionists assessed 7 apps, and the remaining apps were randomly allocated to ensure 4 assessments per app. Intraclass correlation was used to evaluate interrater agreement. Means and standard deviations of scores for each section and each item were calculated.

RESULTS

The top scores for overall quality were obtained by Yazio - Régime et Calories (mean 3.84, SD 0.32), FeelEat (mean 3.71, SD 0.47), and Bonne App (mean 3.65, SD 0.09). Engagement scores ranged from a mean of 1.95 (SD 0.5) for iEatBetter: Journal alimentaire to a mean of 3.85 (SD 0.44) for FeelEat. Functionality scores ranged from a mean of 2.25 (SD 0.54) for Naor to a mean of 4.25 (SD 0.46) for Yazio. Aesthetics scores ranged from a mean of 2.17 (SD 0.34) for Naor to a mean of 3.88 (SD 0.47) for Yazio. Information scores ranged from a mean of 2.38 (SD 0.60) for iEatBetter to a mean of 3.73 (SD 0.29) for Yazio. Subjective quality scores ranged from a mean of 1.13 (SD 0.25) for iEatBetter to a mean of 2.28 (SD 0.88) for Compteur de calories FatSecret. Specificity scores ranged from a mean of 1.38 (SD 0.64) for iEatBetter to a mean of 3.50 (SD 0.91) for FeelEat. The app-specific score was always lower than the subjective quality score, which was always lower than the quality score, which was lower than the rating from the iOS or Android app stores.

CONCLUSIONS

Although prevention and information messages in apps regarding nutritional habits are not scientifically verified before marketing, we found that app quality was good. Subjective quality and specificity were associated with lower ratings. Further investigations are needed to assess whether information from these apps is consistent with recommendations and to determine the long-term impacts of these apps on users.

Agile, Easily Applicable, and Useful eHealth Usability Evaluations: Systematic Review and Expert-Validation

Background  Electronic health (eHealth) usability evaluations of rapidly developed eHealth systems are difficult to accomplish because traditional usability evaluation methods require substantial time in preparation and implementation. This illustrates the growing need for fast, flexible, and cost-effective methods to evaluate the usability of eHealth systems. To address this demand, the present study systematically identified and expert-validated rapidly deployable eHealth usability evaluation methods.

Objective  Identification and prioritization of eHealth usability evaluation methods suitable for agile, easily applicable, and useful eHealth usability evaluations.

Methods  The study design comprised a systematic iterative approach in which expert knowledge was contrasted with findings from literature. Forty-three eHealth usability evaluation methods were systematically identified and assessed regarding their ease of applicability and usefulness through semi-structured expert interviews with 10 European usability experts and systematic literature research. The most appropriate eHealth usability evaluation methods were selected stepwise based on the experts' judgements of their ease of applicability and usefulness.

Results  Of these 43 eHealth usability evaluation methods identified as suitable for agile, easily applicable, and useful eHealth usability evaluations, 10 were recommended by the experts based on their usefulness for rapid eHealth usability evaluations. The three most frequently recommended eHealth usability evaluation methods were Remote User Testing, Expert Review, and Rapid Iterative Test and Evaluation Method. Eleven usability evaluation methods, such as Retrospective Testing, were not recommended for use in rapid eHealth usability evaluations.

Conclusion  We conducted a systematic review and expert-validation to identify rapidly deployable eHealth usability evaluation methods. The comprehensive and evidence-based prioritization of eHealth usability evaluation methods supports faster usability evaluations, and so contributes to the ease-of-use of emerging eHealth systems.

Usability Testing and Technology Acceptance of an mHealth App at the Point of Care During Simulated Pediatric In- and Out-of-Hospital Cardiopulmonary Resuscitations: Study Nested Within 2 Multicenter Randomized Controlled Trials

BACKGROUND

Mobile apps are increasingly being used in various domains of medicine. Few are evidence-based, and their benefits can only be achieved if end users intend to adopt and use them. To date, only a small fraction of mobile apps have published data on their field usability and end user acceptance results, especially in emergency medicine.

OBJECTIVE

This study aims to determine the usability and acceptance of an evidence-based mobile app while safely preparing emergency drugs at the point of care during pediatric in- and out-of-hospital cardiopulmonary resuscitations by frontline caregivers.

METHODS

In 2 multicenter randomized controlled parent trials conducted at 6 pediatric emergency departments from March 1 to December 31, 2017, and 14 emergency medical services from September 3, 2019, to January 21, 2020, the usability and technology acceptance of the PedAMINES (Pediatric Accurate Medication in Emergency Situations) app were evaluated among skilled pediatric emergency nurses and advanced paramedics when preparing continuous infusions of vasoactive drugs and direct intravenous emergency drugs at pediatric dosages during standardized, simulation-based, pediatric in- and out-of-hospital cardiac arrest scenarios, respectively. Usability was measured using the 10-item System Usability Scale. A 26-item technology acceptance self-administered survey (5-point Likert-type scales), adapted from the Unified Theory of Acceptance and Use of Technology model, was used to measure app acceptance and intention to use.

RESULTS

All 100% (128/128) of nurses (crossover trial) and 49.3% (74/150) of paramedics (parallel trial) were assigned to the mobile app. Mean total scores on the System Usability Scale were excellent and reached 89.5 (SD 8.8; 95% CI 88.0-91.1) for nurses and 89.7 (SD 8.7; 95% CI 87.7-91.7) for paramedics. Acceptance of the technology was very good and rated on average >4.5/5 for 5 of the 8 independent constructs evaluated. Only the image construct scored between 3.2 and 3.5 by both participant populations.

CONCLUSIONS

The results provide evidence that dedicated mobile apps can be easy to use and highly accepted at the point of care during in- and out-of-hospital cardiopulmonary resuscitations by frontline emergency caregivers. These findings can contribute to the implementation and valorization of studies aimed at evaluating the usability and acceptance of mobile apps in the field by caregivers, even in critical situations.

TRIAL REGISTRATION

ClinicalTrials.gov NCT03021122; https://clinicaltrials.gov/ct2/show/NCT03021122. ClinicalTrials.gov NCT03921346; https://clinicaltrials.gov/ct2/show/NCT03921346.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.1186/s13063-019-3726-4.

Iterative Development of a Mobile Phone App to Support Community Health Volunteers During Cervical Cancer Screening in Western Kenya: Qualitative Study

BACKGROUND

To achieve the World Health Organization targets for cervical cancer elimination, low- and middle-income countries will need to develop innovative strategies to provide human papillomavirus (HPV)-based screening at a population level. Although mobile health (mHealth) interventions may help realize these goals by filling gaps in electronic specimen tracking and patient education, effective implementation of mHealth interventions is dependent upon context-specific development that is acceptable and usable by the target population. Detailed feedback should be gathered at the design and development stages to yield final products that reflect the needs, desires, and capabilities of target users.

OBJECTIVE

The aim of this study is to develop an mHealth app (mSaada) to support HPV-based screening in partnership with community health volunteers (CHVs) and program planners in western Kenya.

METHODS

A team of student programmers developed a prototype to meet previously identified gaps in screening: patient education, protocol support, data capture, and specimen tracking. The prototype was iteratively developed through 2 waves of in-person working sessions with quantitative (survey) and qualitative (in-depth interview) feedback. Research staff engaged key stakeholders from both urban and rural locations and with varying levels of experience in delivering screening services. During the sessions, participants completed simulation exercises and role-play activities to become familiar with the platform. Once feedback was gathered and synthesized after each wave of in-person data collection, developers implemented changes to improve mSaada functionality.

RESULTS

A total of 18 CHVs and clinicians participated in the in-person sessions. Participants found mSaada useful, easy to use, and would meet the needs of CHVs to provide HPV-based cervical cancer screening (electronic data capture, client education resources, and specimen tracking). They provided key feedback to enhance user experience, workflow, and sustainability. Key changes included altering the appearance of the wireframes, adding translation in additional local languages, changing potentially insensitive figures, alphabetizing lengthy dropdown menus, adding clinically relevant logic checks when entering data, and incorporating the ability to make real time edits to client records. They also made recommendations for additional features that might enhance mSaada's impact at the facility and health system levels, specifically the inclusion of a report-generating tool consistent with the Ministry of Health standards.

CONCLUSIONS

Using a process of iterative feedback with key stakeholders and rapid response from developers, we have developed a mobile app ready for pilot testing in HPV-based screening programs led by CHVs.

A Clinical Data Management System for Diabetes Clinical Trials

BACKGROUND

The use of novel medications and methods to prevent, diagnose, treat, and manage diabetes requires confirmation of safety and efficacy in a well-designed study prior to widespread adoption. Diabetes clinical trials are the studies that examine these issues. The aim of the present study was to develop a web-based system for data management in diabetes clinical trials.

METHODS

The present research was a mixed-methods study conducted in 2019. To identify the required data elements and functions to develop the system, 60 researchers completed a questionnaire. The designed system was evaluated using two methods. The usability of the system was initially evaluated by a group of researchers (n = 6) using the think-aloud method, and after system improvement, the system functions were evaluated by other researchers (n = 30) using a questionnaire.

RESULTS

The main data elements which were required to develop a case report form included "study data," "participant's personal data," and "clinical data." The functional requirements of the system were "managing the study," "creating case report forms," "data management," "data quality control," and "data security and confidentiality." After using the system, researchers rated the system functions at a "good" level (6.3 ± 0.73) on a seven-point Likert scale.

CONCLUSION

Given the complexity of the data management processes in diabetes clinical trials and the widespread use of information technologies in research, the use of clinical data management systems in diabetes clinical trials seems inevitable. The system developed in the current study can facilitate and improve the process of creating and managing case report forms as well as collecting data in diabetes clinical trials.

Chinese Version of the mHealth App Usability Questionnaire: Cross-Cultural Adaptation and Validation

Background

Mobile health (mHealth) apps have shown the advantages of improving medication compliance, saving time required for diagnosis and treatment, reducing medical expenses, etc. The World Health Organization (WHO) has recommended that mHealth apps should be evaluated prior to their implementation to ensure their accuracy in data analysis.

Objective

This study aimed to translate the patient version of the interactive mHealth app usability questionnaire (MAUQ) into Chinese, and to conduct cross-cultural adaptation and reliability and validity tests.

Methods

The Brislin’s translation model was used in this study. The cross-cultural adaptation was performed according to experts’ comments and the results of prediction test. The convenience sampling method was utilized to investigate 346 patients who used the “Good Doctor” (“Good Doctor” is the most popular mHealth app in China), and the reliability and validity of the questionnaire were evaluated as well.

Results

After translation and cross-cultural adaptation, there were a total of 21 items and 3 dimensions: usability and satisfaction (8 items), system information arrangement (6 items), and efficiency (7 items). The content validity index was determined to be 0.952, indicating that the 21 items used to evaluate the usability of the Chinese version of the MAUQ were well correlated. The Cronbach’s α coefficient of the total questionnaire was 0.912, which revealed that the questionnaire had a high internal consistency. The values of test-retest reliability and split-half reliability of the Chinese version of the MAUQ were 0.869 and 0.701, respectively, representing that the questionnaire had a good stability.

Conclusion

The translated questionnaire has good reliability and validity in the context of Chinese culture, and it could be used as a usability testing tool for the patient version of interactive mHealth apps.

Improving usability benchmarking for the eHealth domain: The development of the eHealth UsaBility Benchmarking instrument (HUBBI)

Background

Currently, most usability benchmarking tools used within the eHealth domain are based on re-classifications of old usability frameworks or generic usability surveys. This makes them outdated and not well suited for the eHealth domain. Recently, a new ontology of usability factors was developed for the eHealth domain. It consists of eight categories: Basic System Performance (BSP), Task-Technology Fit (TTF), Accessibility (ACC), Interface Design (ID), Navigation & Structure (NS), Information & Terminology (IT), Guidance & Support (GS) and Satisfaction (SAT).

Objective

The goal of this study is to develop a new usability benchmarking tool for eHealth, the eHealth UsaBility Benchmarking Instrument (HUBBI), that is based on a new ontology of usability factors for eHealth.

Methods

First, a large item pool was generated containing 66 items. Then, an online usability test was conducted, using the case study of a Dutch website for general health advice. Participants had to perform three tasks on the website, after which they completed the HUBBI. Using Partial Least Squares Structural Equation Modelling (PLS-SEM), we identified the items that assess each factor best and that, together, make up the HUBBI.

Results

A total of 148 persons participated. Our selection of items resulted in a shortened version of the HUBBI, containing 18 items. The category Accessibility is not included in the final version, due to the wide range of eHealth services and their heterogeneous populations. This creates a constantly different role of Accessibility, which is a problem for a uniform benchmarking tool.,

Conclusions

The HUBBI is a new and comprehensive usability benchmarking tool for the eHealth domain. It assesses usability on seven domains (BSP, TTF, ID, NS, IT, GS, SAT) in which a score per domain is generated. This can help eHealth developers to quickly determine which areas of the eHealth system’s usability need to be optimized.

User-Centered Design of A Novel Risk Prediction Behavior Change Tool Augmented With an Artificial Intelligence Engine (MyDiabetesIQ): A Sociotechnical Systems Approach

Background

Diabetes and its complications account for 10% of annual health care spending in the United Kingdom. Digital health care interventions (DHIs) can provide scalable care, fostering diabetes self-management and reducing the risk of complications. Tailorability (providing personalized interventions) and usability are key to DHI engagement/effectiveness. User-centered design of DHIs (aligning features to end users’ needs) can generate more usable interventions, avoiding unintended consequences and improving user engagement.

Objective

MyDiabetesIQ (MDIQ) is an artificial intelligence engine intended to predict users’ diabetes complications risk. It will underpin a user interface in which users will alter lifestyle parameters to see the impact on their future risks. MDIQ will link to an existing DHI, My Diabetes My Way (MDMW). We describe the user-centered design of the user interface of MDIQ as informed by human factors engineering.

Methods

Current users of MDMW were invited to take part in focus groups to gather their insights about users being shown their likelihood of developing diabetes-related complications and any risks they perceived from using MDIQ. Findings from focus groups informed the development of a prototype MDIQ interface, which was then user-tested through the “think aloud” method, in which users speak aloud about their thoughts/impressions while performing prescribed tasks. Focus group and think aloud transcripts were analyzed thematically, using a combination of inductive and deductive analysis. For think aloud data, a sociotechnical model was used as a framework for thematic analysis.

Results

Focus group participants (n=8) felt that some users could become anxious when shown their future complications risks. They highlighted the importance of easy navigation, jargon avoidance, and the use of positive/encouraging language. User testing of the prototype site through think aloud sessions (n=7) highlighted several usability issues. Issues included confusing visual cues and confusion over whether user-updated information fed back to health care teams. Some issues could be compounded for users with limited digital skills. Results from the focus groups and think aloud workshops were used in the development of a live MDIQ platform.

Conclusions

Acting on the input of end users at each iterative stage of a digital tool’s development can help to prioritize users throughout the design process, ensuring the alignment of DHI features with user needs. The use of the sociotechnical framework encouraged the consideration of interactions between different sociotechnical dimensions in finding solutions to issues, for example, avoiding the exclusion of users with limited digital skills. Based on user feedback, the tool could scaffold good goal setting, allowing users to balance their palatable future complications risk against acceptable lifestyle changes. Optimal control of diabetes relies heavily on self-management. Tools such as MDMW/ MDIQ can offer personalized support for self-management alongside access to users’ electronic health records, potentially helping to delay or reduce long-term complications, thereby providing significant reductions in health care costs.

Usability and Engagement Testing of mHealth Apps in Paediatric Obesity: A Narrative Review of Current Literature

Mobile health (mHealth) platforms have become increasingly popular for delivering health interventions in recent years and particularly in light of the COVID-19 pandemic. Childhood obesity treatment is an area where mHealth interventions may be useful due to the multidisciplinary nature of interventions and the need for long-term care. Many mHealth apps targeting youth exist but the evidence base underpinning the methods for assessing technical usability, user engagement and user satisfaction of such apps with target end-users or among clinical populations is unclear, including for those aimed at paediatric overweight and obesity management. This review aims to examine the current literature and provide an overview of the scientific methods employed to test usability and engagement with mHealth apps in children and adolescents with obesity. A narrative literature review was undertaken following a systematic search. Four academic databases were searched. Inclusion criteria were studies describing the usability of mHealth interventions for childhood obesity treatment. Following the application of inclusion and exclusion criteria, fifty-nine articles were included for full-text review, and seven studies met the criteria for usability and engagement in a clinical paediatric population with obesity. Six apps were tested for usability and one for engagement in childhood obesity treatment. Sample sizes ranged from 6-1120 participants. The included studies reported several heterogenous measurement instruments, data collection approaches, and outcomes. Recommendations for future research include the standardization and validation of instruments to measure usability and engagement within mHealth studies in this population.

Methods for Human-Centered eHealth Development: Narrative Scoping Review

Background

Thorough holistic development of eHealth can contribute to a good fit among the technology, its users, and the context. However, despite the availability of frameworks, not much is known about specific research activities for different aims, phases, and settings. This results in researchers having to reinvent the wheel. Consequently, there is a need to synthesize existing knowledge on research activities for participatory eHealth development processes.

Objective

The 3 main goals of this review are to create an overview of the development strategies used in studies based on the CeHRes (Center for eHealth Research) Roadmap, create an overview of the goals for which these methods can be used, and provide insight into the lessons learned about these methods.

Methods

We included eHealth development studies that were based on the phases and/or principles of the CeHRes Roadmap. This framework was selected because of its focus on participatory, iterative eHealth design in context and to limit the scope of this review. Data were extracted about the type of strategy used, rationale for using the strategy, research questions, and reported information on lessons learned. The most frequently mentioned lessons learned were summarized using a narrative, inductive approach.

Results

In the included 160 papers, a distinction was made between overarching development methods (n=10) and products (n=7). Methods are used to gather new data, whereas products can be used to synthesize previously collected data and support the collection of new data. The identified methods were focus groups, interviews, questionnaires, usability tests, literature studies, desk research, log data analyses, card sorting, Delphi studies, and experience sampling. The identified products were prototypes, requirements, stakeholder maps, values, behavior change strategies, personas, and business models. Examples of how these methods and products were applied in the development process and information about lessons learned were provided.

Conclusions

This study shows that there is a plethora of methods and products that can be used at different points in the development process and in different settings. To do justice to the complexity of eHealth development, it seems that multiple strategies should be combined. In addition, we found no evidence for an optimal single step-by-step approach to develop eHealth. Rather, researchers need to select the most suitable research methods for their research objectives, the context in which data are collected, and the characteristics of the participants. This study serves as a first step toward creating a toolkit to support researchers in applying the CeHRes Roadmap to practice. In this way, they can shape the most suitable and efficient eHealth development process.

The most used questionnaires for evaluating satisfaction, usability, acceptance, and quality outcomes of mobile health

Background

Various questionnaires are used for evaluating satisfaction, usability, acceptance, and quality outcomes of mobile health (mHealth) services. Using the best one to meet the needs of an mHealth study is a challenge for researchers. Therefore, this study aimed to review and determine the frequently used questionnaires for evaluating the mentioned outcomes of mHealth services.

Methods

The PubMed database was searched for conducting this review in April 2021. Papers that used a referenced questionnaire to evaluate the satisfaction, usability, acceptance, or quality outcomes of mHealth were included. The first author’s name, year of publication, evaluation outcome, and evaluation questionnaire were extracted from relevant papers. Data were analyzed using descriptive statistics.

Results

In total, 247 papers were included in the study. Questionnaires were used for usability (40%), quality (34.5%), acceptance (8.5%), and satisfaction (4%) outcomes, respectively. System usability scale (36.5%), mobile application rating scale (35.5%), post study system usability questionnaire (6%), user mobile application rating scale (5%), technology acceptance model (4.5%), computer system usability questionnaire (2.5%), net promoter score (2%), health information technology usability evaluation scale (2%), the usefulness, satisfaction, and ease of use (1.5%), client satisfaction questionnaire (1.5%), unified theory of acceptance and use of technology (1.5%), questionnaire for user interaction satisfaction (1%), user experience questionnaire (1%), and after-scenario questionnaire (1%) were the most used questionnaires, respectively.

Conclusion

Despite the existence of special questionnaires for evaluating several outcomes of mHealth, general questionnaires with fewer items and higher reliability have been used more frequently. Researchers should pay more attention to questionnaires with a goal-based design.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-022-01764-2.

Smart Assistive Technology for Cooking for People With Cognitive Impairments Following a Traumatic Brain Injury: User Experience Study

Background

User experience (UX), including usability, should be formally assessed multiple times throughout the development process to optimize the acceptability and integration of a new technology before implementing it within the home environment of people living with cognitive impairments.

Objective

The aim of this study is to identify UX issues, notably usability issues, and factors to consider for the future implementation of the COOK (Cognitive Orthosis for Cooking) within the home of individuals with traumatic brain injury (TBI) to identify modifications to improve the technology.

Methods

This study comprised two rounds of UX evaluations, including extensive usability testing, which were completed in a laboratory context: 3 sessions with 5 experts and, after improvement of COOK, 2 sessions with 10 participants with TBI. Each session included the use of scenarios and questionnaires on UX and usability.

Results

Both rounds demonstrated good usability outcomes and hedonic qualities. Various usability issues were identified by participants, such as navigation inconsistencies, technical bugs, and the need for more feedback. Factors to consider in the future implementation of COOK were also mentioned by participants with TBI, including environmental (eg, space available and presence of pets) and personal factors (eg, level of comfort with technology, presence of visual deficits, and preferences).

Conclusions

By evaluating UX, including usability, various times throughout the development process and including experts and end users, our research team was able to develop a technology that was perceived as usable, pleasant, and well-designed. This research is an example of how and when people with cognitive impairments (ie, people with TBI) can be involved in evaluating the UX of new technology.

Encouraging Behavior Changes and Preventing Cardiovascular Diseases Using the Prevent Connect Mobile Health App: Conception and Evaluation of App Quality

Background

Cardiovascular diseases are a major cause of death worldwide. Mobile health apps could help in preventing cardiovascular diseases by improving modifiable risk factors such as eating habits, physical activity levels, and alcohol or tobacco consumption.

Objective

The aim of this study was to design a mobile health app, Prevent Connect, and to assess its quality for (1) assessing patient behavior for 4 cardiovascular risk factors (unhealthy eating, sedentary lifestyle, alcohol, and tobacco consumption) and (2) suggesting personalized recommendations and mobile health interventions for risky behaviors.

Methods

The knowledge base of the app is based on French national recommendations for healthy eating, physical activity, and limiting alcohol and tobacco consumption. It contains a list of patient behaviors and related personalized recommendations and digital health interventions. The interface was designed according to usability principles. Its quality was assessed by a panel of 52 users in a 5-step process: completion of the demographic form, visualization of a short presentation of the app, testing of the app, completion of the user version of the Mobile App Rating Scale (uMARS), and an open group discussion.

Results

This app assesses patient behaviors through specific questionnaires about 4 risk factors (unhealthy eating, sedentary lifestyle, alcohol, and tobacco consumption) and suggests personalized recommendations and digital health interventions for improving behavior. The app was deemed to be of good quality, with a mean uMARS quality score of 4 on a 5-point Likert scale. The functionality and information content of the app were particularly appreciated, with a mean uMARS score above 4. Almost all the study participants appreciated the navigation system and found the app easy to use. More than three-quarters of the study participants found the app content relevant, concise, and comprehensive. The aesthetics and the engagement of the app were also appreciated (uMARS score, 3.7). Overall, 80% (42/52) of the study participants declared that the app helped them to become aware of the importance of addressing health behavior, and 65% (34/52) said that the app helped motivate them to change lifestyle habits.

Conclusions

The app assessed the risky behaviors of the patients and delivered personalized recommendations and digital health interventions for multiple risk factors. The quality of the app was considered to be good, but the impact of the app on behavior changes is yet to be demonstrated and will be assessed in further studies.

Hybrid-Pattern Recognition Modeling with Arrhythmia Signal Processing for Ubiquitous Health Management

We established a web-based ubiquitous health management (UHM) system, "ECG4UHM", for processing ECG signals with AI-enabled models to recognize hybrid arrhythmia patterns, including atrial premature atrial complex (APC), atrial fibrillation (AFib), ventricular premature complex (VPC), and ventricular tachycardia (VT), versus normal sinus rhythm (NSR). The analytical model coupled machine learning methods, such as multiple layer perceptron (MLP), random forest (RF), support vector machine (SVM), and naive Bayes (NB), to process the hybrid patterns of four arrhythmia symptoms for AI computation. The data pre-processing used Hilbert-Huang transform (HHT) with empirical mode decomposition to calculate ECGs' intrinsic mode functions (IMFs). The area centroids of the IMFs' marginal Hilbert spectrum were suggested as the HHT-based features. We engaged the MATLAB^TM compiler and runtime server in the ECG4UHM to build the recognition modules for driving AI computation to identify the arrhythmia symptoms. The modeling extracted the crucial data sets from the MIT-BIH arrhythmia open database. The validated models, including the premature pattern (i.e., APC-VPC) and the fibril-rapid pattern (i.e., AFib-VT) against NSR, could reach the best area under the curve (AUC) of the receiver operating characteristic (ROC) of approximately 0.99. The models for all hybrid patterns, without VPC versus AFib and VT, achieved an average accuracy of approximately 90%. With the prediction test, the respective AUCs of the NSR and APC versus the AFib, VPC, and VT were 0.94 and 0.93 for the RF and SVM on average. The average accuracy and the AUC of the MLP, RF, and SVM models for APC-VT reached the value of 0.98. The self-developed system with AI computation modeling can be the backend of the intelligent social-health system that can recognize hybrid arrhythmia patterns in the UHM and home-isolated cares.

InCoPE-App: Study protocol to examine usability and effectiveness of an individualized, tablet-based multidomain exercise program for institutionalized people with dementia delivered by nursing assistants. (Preprint)

Background

The COVID-19 pandemic has had drastic consequences on everyday life in nursing homes. Limited personnel resources and modified hygiene and safety measures (eg, no external exercise instructors, no group settings) have often led to interrupted physical exercise treatments. As a consequence, people with dementia benefiting from individualized exercise programs are affected by the pandemic’s impact.

Objective

Our goal is to develop an easily applicable mobile application (Individualized Cognitive and Physical Exercise [InCoPE] app) allowing nursing assistants to test cognitive function and physical performance and subsequently train people with dementia through a multidomain, individualized exercise program.

Methods

We will evaluate the usability and effectiveness of the InCoPE-App by applying a mixed method design. Nursing assistants will use the InCoPE-App for 18 weeks to assess the cognitive function and physical performance of 44 people with dementia every 3 weeks and apply the individualized exercise program. We will record overall usability using questionnaires (eg, Post-Study System Usability and ISONORM 9241/10), log events, and interviews. Perceived hedonic and pragmatic quality will be assessed using the AttrakDiff questionnaire. Effectiveness will be evaluated by considering changes in quality of life as well as cognitive function and physical performance between before and after the program.

Results

Enrollment into the study will be completed in the first half of 2022. We expect an improvement in the quality of life of people with dementia accompanied by improvements in cognitive function and physical performance. The usability of the InCoPE-App is expected to be rated well by nursing assistants.

Conclusions

To date, there is no scientifically evaluated app available that enables nursing assistants without expertise in sports science to deliver an individualized exercise program among people with dementia. A highly usable and effective InCoPE-App allows nursing assistants to test cognitive function and physical performance of people with dementia and, based thereon, select and deliver an appropriate individualized exercise program based on the cognitive and physical status of an individual, even in times of a pandemic.

Trial Registration

German Register of Clinical Trials DRKS00024069; https://www.drks.de/drks_web/navigate.do?navigationId=trial.HTML&TRIAL_ID=DRKS00024069

International Registered Report Identifier (IRRID)

DERR1-10.2196/36247

Continuous User Experience Monitoring of a Patient-Completed Preoperative Assessment System in the United Kingdom: Cross-sectional Study

BACKGROUND

Anesthetic preoperative assessment (POA) is now a common part of the surgical care pathway, and guidelines support its routine use. MyPreOp (Ultramed Ltd) is a web-based POA system that enables remote assessments. Usability is a key factor in the success of digital health solutions.

OBJECTIVE

This study aims to assess the usability of the MyPreOp system through patient feedback, investigate the amount of time it took for patients to complete the POA questionnaire and the factors that influenced completion time, and explore the effect on completion times of implementing a validated eHealth usability scale, as compared to using a simple but unvalidated usability evaluation scale, and to test the feasibility of administering a more detailed usability evaluation scale in a staggered manner so as not to unduly increase completion times.

METHODS

In this cross-sectional study, anonymized data sets were extracted from the MyPreOp system. The participants were adults (aged ≥18 years), scheduled for nonurgent surgical procedures performed in hospitals in the United Kingdom, who gave consent for their anonymized data to be analyzed. Data collected included age, gender, American Society of Anesthesiology (ASA) physical classification status, and completion time. Two user experience evaluations were used: in Phase 1, 2 questions asking about overall experience and ease of use, and in Phase 2, a previously validated usability questionnaire, with its 20 questions equally distributed among 5 succeeding patient cohorts. There were 2593 respondents in total (Phase 1: n=1193; Phase 2: n=1400). The median age of the participants was 46 years, and 1520 (58.62%) of the 2593 respondents were female. End points measured were the median completion times in Phase I and Phase II. The data were collected by extracting a subset of records from the database and exported to a spreadsheet for analysis (Excel, Microsoft Corporation). The data were analyzed for differences in completion times between Phase I and Phase II, as well as for differences between age groups, genders, and ASA classifications.

RESULTS

MyPreOp scored well in usability in both phases. In Phase 1, 81.64% (974/1193) of respondents had a good or better experience, and 93.8% (1119/1193) found it easy to use. The usability rating in Phase 2 was 4.13 out of a maximum of 5, indicating high usability. The median completion time was 40.4 minutes. The implementation of the longer usability evaluation scale in Phase 2 did not negatively impact the completion times. Age and ASA physical status were found to be moderately associated with increased completion times.

CONCLUSIONS

MyPreOp rates high in both user experience and usability. The method of dividing the questionnaire into 5 blocks is valid and does not negatively affect completion times. Further research into the factors affecting completion time is recommended.

Smartphone application-based follow-up care of patients after bariatric surgery: A mixed-method study of usability

OBJECTIVE

Due to potential short-term and long-term complications, adequate lifelong follow-up is crucial for patients after bariatric surgery. However, compliance with postoperative follow-up is poor despite clear national and international guidelines. This feasibility trial aimed to evaluate the usability of a smartphone application-based follow-up program in patients after bariatric surgery.

METHODS

Patients were included after having undergone a primary bariatric surgery. Instead of attending meetings in our outpatient clinic, they were followed up using a smartphone application. After 6 months, the System Usability Scale (SUS) was used to measure participant perception. Additional interviews were performed to gain more insight into the usability of the app.

RESULTS

Between August 2020 and February 2021, 52 patients met the inclusion criteria and agreed to participate in the study, of whom 5 (9.6%) dropped out of the study. At the time of analysis, 31 patients have been followed up for 6 months, among whom 26 patients completed the SUS questionnaire. The mean overall score of SUS is 82.1 ± 19.8. In the qualitative analysis, participants were generally positive about the follow-up care and found it easy to use, despite some minor technical problems.

CONCLUSION

Our smartphone app-based follow-up program was proven to be effective in the aftercare succeeding bariatric surgery. Our data indicates that the satisfaction, efficiency, learnability, and ease of use of the smartphone application were coherent in the acceptance and use of mobile technology by patients.

Assessments of a novel digital follow-up tool Rehabkompassen® to identify rehabilitation needs among stroke patients in an outpatient setting

Introduction

It remains a huge challenge to identify individual rehabilitation needs in a time-efficient manner for providing patient-tailored rehabilitation during the continuum of stroke care. We have recently demonstrated the usefulness of a paper-version Rehab-Compass as a follow-up tool. The aim of the current study was to develop a digital version of the Rehab-Compass and evaluate its usability and feasibility.

Methods

The novel digital tool Rehabkompassen® was developed by an iterative and participatory design process. Patients' rehabilitation needs were visualized by the tool and used before, during, and after the consultation. The usability and feasibility of the tool was assessed by task completion rate, the System Usability Scale, and satisfaction questionnaires among 2 physicians and 24 adult stroke patients in an outpatient clinical setting.

Results

Rehabkompassen® identified and graphically visualized a panoramic view of the stroke patients' multidimensional needs in individual- and group levels. The instrument appeared to be feasible and time efficient in clinical use with a 100% overall task completion rate for both patients and physicians. A majority of the patients reported that it was very easy or fairly easy to answer the digital questionnaires and to understand their own digital Rehab-Compass graph. Two physicians reported a high mean score on the System Usability Scale (95/100) and were positive about using the tool in the future.

Conclusions

The current results indicated that Rehabkompassen® was a feasible, useful, and time-saving follow-up tool for the identification of rehabilitation needs among stroke survivors in the post-acute continuum of care after stroke. Further research is needed to evaluate the efficacy of the digital instrument among stroke patients.

Assessing the usability and acceptability of Face IT@home: an online CBT intervention for people with visible differences

To increase access to support, an online psychosocial support tool for adults with visible differences was adapted for use without referral or supervision. This intervention combines a cognitive behavioural and social skills model of support. This study aimed to assess the usability and acceptability of Face IT@home as a self-help intervention. Eighty-one participants were recruited (32 with visible differences). Stage 1 included 14 participants (11 female, all with visible differences) who viewed two sessions of Face IT@home and undertook a semi-structured telephone interview. Stage 2 consisted of 14 think-aloud sessions (13 female, none with visible differences) with participants, supervised by researchers. Stage 3 employed 53 participants (47 female; 19 with visible differences), to view one session of Face IT@home and complete an online survey to evaluate usability and acceptability. User interviews, think-aloud studies and questionnaires identified usability and acceptability factors of Face IT@home that make it fit for purpose as a self-help tool. Participants suggested some changes to the Face IT@home program to improve usability. Participants reported that Face IT@home was a useful tool for people with visible differences and could be effective. The CBT-based model was considered a useful approach to addressing psychosocial concerns. The online self-help format will increase access to psychological support for adults with visible differences.

Key learning aims

(1)

A roadmap for the development and evaluation of the eHealthResp online course

Background

Inappropriate antibiotic use constitutes one of the most concerning public health issues, being one of the main causes of antibiotic resistance. Hence, to tackle this issue, it is important to encourage the development of educational interventions for health practitioners, namely by using digital health tools. This study focuses on the description of the development and validation process of the eHealthResp online course, a web platform directed to physicians and pharmacists, with the overall goal of improving antibiotic use for respiratory tract infections, along with the assessment of its usability.

Methods

The eHealthResp platform and the courses, developed with a user-centered design and based on Wordpress and MySQL, were based on a previously developed online course. A questionnaire to assess the usability was distributed among physicians (n = 6) and pharmacists (n = 6). Based on the obtained results, statistical analyses were conducted to calculate the usability score and appraise the design of the online course, as well as to compare the overall scores attributed by both groups. Further qualitative comments provided by the participants have also been analyzed.

Results

The eHealthResp contains two online courses directed to physicians and pharmacists aiming to aid in the management of respiratory tract infections. The average usability score of the eHealthResp online courses for physicians and pharmacists was of 78.33 (±11.57, 95%CI), and 83.75 (±15.90, 95%CI), respectively. Qualitative feedback emphasized the usefulness of the course, including overall positive reviews regarding user-friendliness and consistency.

Conclusions

This study led us to conclude that the eHealthResp online course is not recognized as a complex web platform, as both qualitative and quantitative feedback obtained were globally positive.

Adolescents’ opinions on the use of a smartphone application as an oral health education tool: A qualitative study

Developing health-related smartphone applications for oral health education should consider the end-user's perspectives to ensure they will be usable. This study aimed to explore the opinions of secondary school students in Selangor, Malaysia regarding the use of a smartphone application for oral health education and to identify the features for an oral health education smartphone application from the perspectives of adolescents. Focus group discussions were conducted among Form Two (14-year-old) and Form Four (16-year-old) students from selected government secondary schools in Selangor utilising a semi-structured topic guide until data saturation was achieved. Data were analysed using framework analysis. Ten focus group discussions were conducted involving 77 participants. Mixed opinions were recorded on the use of health-related smartphone applications for oral health education. The preferred features in a health-related smartphone application are disease detection, have games and rewards, educational and fun, access to a dentist, reminders, and user-friendliness. Adolescents are aware of the positive aspect of using health-related smartphone applications for oral health education; however, they are wary of the need to install one. Nevertheless, identifying adolescents’ preferred features of an oral health education app is the first step in developing an application tailored to their needs. Smartphone application could be a timely strategy to improve oral health education delivery and behaviour improvement for this age group.

Enhancing System Acceptance through User-Centred Design: Integrating Patient Generated Wellness Data

Gestational diabetes mellitus (GDM) is a condition that appears during pregnancy and is expected to be a temporary one. While patients are encouraged to manage it themselves, research findings indicate that GDM may negatively affect the foetus; in addition, there is an increased risk of women with GDM subsequently developing Type 2 diabetes. To alleviate the risks, women with GDM are advised to maintain a record of their diet and blood glucose levels and to attend regular clinical reviews. Rather than using a paper diary, women with GDM can maintain a record of their blood glucose level readings and other relevant data using a wellness mobile application (app). However, such apps are developed for general use and may not meet the specific needs of clinical staff (physicians, dietitians, obstetricians and midwives) involved in managing GDM; for example, an app may record glucose readings but not the details of a meal taken before or after the glucose reading. Second, the apps do not permanently store the data generated by the patient and do not support the transfer of these data to a clinical system or information portal. The mobile health (mHealth) system designed and developed in this research allows one to integrate different types of user generated wellness data into a centralised database. A user-centered design (UCD) approach informed by the technology acceptance model (TAM) was adopted. This paper investigates and evaluates the effectiveness of the approach with regard to facilitating system acceptance and future adoption through an early focus on enhancing system usefulness and ease of use. The functional system requirements of the proposed system were refined through a series of interviews with the perspective of clinical users; ease-of-use and usability issues were resolved through 'think aloud' sessions with clinicians and GDM patients.

Online self-exclusion from multiple gambling venues: Stakeholder co-design of a usable and acceptable self-directed website

Gambling self-exclusion programs are under-utilised and barriers to entry include shame and embarrassment with face-to-face registration, and complex and effortful procedures. The current study aimed to facilitate self-exclusion from gambling venues via an online self-directed website. A co-design approach was used to elicit key stakeholders' perspectives on required website features, functionality, and to identify variables potentially impacting on development and implementation. Semi-structured focus groups and interviews were conducted across four stakeholders (N = 25): self-exclusion end users (consumers, n = 5), gambling counsellors (n = 7), venue staff (n = 6), and policy makers (n = 7). Overall, stakeholder perspectives were consistent with content analysis indicating the importance of website user-friendliness, flexibility, supportiveness, and trustworthiness. Importantly, these attributes were linked to target end users': perceived vulnerabilities, diverse backgrounds and individual expectations. Participants believed that the entire self-exclusion process should be conducted online, including identity verification, whilst expecting high-level data security measures to protect their personal privacy. A separate webpage was also suggested containing relevant information and links to additional help services, such as counselling. This study describes an adaptable co-design framework for developing a usable and acceptable self-exclusion website. Future studies should empirically test system usability and acceptability to refine and maximise system uptake upon implementation. Findings may have broader implications for digital health intervention design.

Upper limb rehabilitation system based on virtual reality for breast cancer patients: Development and usability study

Background

Functional exercise is crucial for breast cancer patients after surgery, and the use of virtual reality technology to assist patients with postoperative upper limb functional rehabilitation has gradually attracted the attention of researchers. However, the usability of the developed rehabilitation system is still unknown to a large extent. The purpose of this study was to develop a virtual reality upper limb rehabilitation system for patients after breast cancer surgery and to explore its usability.

Methods

We built a multidisciplinary team based on virtual reality and human-computer interaction technology and designed and developed an upper limb function rehabilitation system for breast cancer patients after surgery. Breast cancer patients were recruited from a grade III-a general hospital in Changchun city for the experiment. We used the System Usability Scale to evaluate the system availability, the Presence Questionnaire scale to measure the immersive virtual reality scene, and the Simulator Sickness Questionnaire subjective measurement scale for simulator sickness symptoms.

Results

This upper limb rehabilitation system hardware consisted of Head-mounted Display, a control handle and notebook computers. The software consisted of rehabilitation exercises and game modules. A total of 15 patients were tested on this system, all of whom were female. The mean age was 54.73±7.78 years, and no patients were excluded from the experiment because of adverse reactions such as dizziness and vomiting. The System Usability Scale score was 90.50±5.69, the Presence Questionnaire score was 113.40±9.58, the Simulator Sickness Questionnaire-nausea score was 0.93±1.16, the Simulator Sickness Questionnaire-oculomotor score was 0.80±1.27, the Simulator Sickness Questionnaire-disorientation score was 0.80±1.27, and the Simulator Sickness Questionnaire total score was 2.53±3.40.

Conclusions

This study fills in the blanks regarding the upper limb rehabilitation of breast cancer patients based on virtual reality technology system usability research. As the starting point of research in the future, we will improve the system’s function and design strictly randomized controlled trials, using larger samples in the promotion, to evaluate its application in breast cancer patients with upper limbs and other physiological functions and the feasibility and effects of rehabilitation.

An Intelligent Individualized Cardiovascular App for Risk Elimination (iCARE) for Individuals With Coronary Heart Disease: Development and Usability Testing Analysis

Background

Death and disability from coronary heart disease (CHD) can be largely reduced by improving risk factor management. However, adhering to evidence-based recommendations is challenging and requires interventions at the level of the patient, provider, and health system.

Objective

The aim of this study was to develop an Intelligent Individualized Cardiovascular App for Risk Elimination (iCARE) to facilitate adherence to health behaviors and preventive medications, and to test the usability of iCARE.

Methods

We developed iCARE based on a user-centered design approach, which included 4 phases: (1) function design, (2) iterative design, (3) expert inspections and walkthroughs of the prototypes, and (4) usability testing with end users. The usability testing of iCARE included 2 stages: stage I, which included a task analysis and a usability evaluation (January to March 2019) of the iCARE patient app using the modified Health Information Technology Usability Survey (Health-ITUES); and stage II (June 2020), which used the Health-ITUES among end users who used the app for 6 months. The end users were individuals with a confirmed diagnosis of CHD from 2 university-affiliated hospitals in Beijing, China.

Results

iCARE consists of a patient app, a care provider app, and a cloud platform. It has a set of algorithms that trigger tailored feedback and can send individualized interventions based on data from initial assessment and health monitoring via manual entry or wearable devices. For stage I usability testing, 88 hospitalized patients (72% [63/88] male; mean age 60 [SD 9.9] years) with CHD were included in the study. The mean score of the usability testing was 90.1 (interquartile range 83.3-99.0). Among enrolled participants, 90% (79/88) were satisfied with iCARE; 94% (83/88) and 82% (72/88) reported that iCARE was useful and easy to use, respectively. For stage II usability testing, 61 individuals with CHD (85% [52/61] male; mean age 53 [SD 8.2] years) who were from an intervention arm and used iCARE for at least six months were included. The mean total score on usability testing based on the questionnaire was 89.0 (interquartile distance: 77.0-99.5). Among enrolled participants, 89% (54/61) were satisfied with the use of iCARE, 93% (57/61) perceived it as useful, and 70% (43/61) as easy to use.

Conclusions

This study developed an intelligent, individualized, evidence-based, and theory-driven app (iCARE) to improve patients’ adherence to health behaviors and medication management. iCARE was identified to be highly acceptable, useful, and easy to use among individuals with a diagnosis of CHD.

Trial Registration

Chinese Clinical Trial Registry ChiCTR-INR-16010242; https://tinyurl.com/2p8bkrew

Exploring the relationship between the usability of a goal-oriented mobile health application and non-usage attrition in patients with multimorbidity: A blended data analysis approach

Background

Mobile health applications are increasingly used to support the delivery of health care services to a variety of patients. Based on data obtained from a pragmatic trial of the electronic Patient Reported Outcome (ePRO) app designed to support goal-oriented care primary care, this study aims to (1) examine how patient-reported usability changed over the one-year intervention period, and (2) explore participant attrition rate of the electronic Patient Reported Outcome app over one year study period.

Methods

We performed a secondary analysis of 44 older adults with complex chronic needs enrolled in the electronic Patient Reported Outcome-digital health intervention. App usage and attrition were measured using device-generated usage logs; usability was measured using the patient-reported post-study system usability questionnaire collected at 3, 6, 9, and 12 months. Research memos were used to interpret potential contextual contributing factors to patients' overall usage and usability score pattern. A data triangulation method of both quantitative and qualitative data was used to analyze and interpret study findings.

Results

While there was gradual attrition in the use of the ePRO app, patients' usability scores remained consistent throughout the study period. Qualitative memos suggested patients' encounters with technical difficulties and relationship dynamics with primary providers influenced patients' adherence to the ePRO app.

Conclusion

This study highlights that the patient-provider relationship is a key determining factor that influences complex patients' continued engagement with a Mobile health app. The finding calls attention to the measurement of usability of a Mobile health app, its impact on attrition, and contributing factors that influence patients' attrition. Trial registration: Clinicaltrials.gov Identified NCT02917954.

Patient Experience and Satisfaction with an e-Health Care Management Application for Inflammatory Bowel Diseases

Background: Rising healthcare expenditures have been partially attributed to suboptimal management of inflammatory bowel diseases (IBD). Electronic health interventions may help improve care management for IBD patients, but there is a need to better understand patient perspectives on these emerging technologies. Aims: The primary aim was to evaluate patient satisfaction and experience with the UCLA eIBD mobile application, an integrative care management platform with disease activity monitoring tools and educational modules. The secondary objective was to capture patient feedback on how to improve the mobile application. Methods: We surveyed IBD patients treated at the UCLA Center for Inflammatory Bowel Diseases. The patient experience survey assessed the patients’ overall satisfaction with the application, perception of health outcomes after participation in the program, and feedback on educational modules as well as areas for application improvement. Results: 50 patients were included. The responses indicated that the patients were greatly satisfied with the ease of patient–provider communication within the application and appointment scheduling features (68%). A majority of respondents (54%) also reported that program participation resulted in improved perception of disease control and quality of life. Lastly, a majority of participants (79%) would recommend this application to others. Conclusions: Mobile tools such as UCLA eIBD have promising implications for integration into patients’ daily lives. This patient satisfaction study suggests the feasibility of using this mobile application by patients and providers. We further showed that UCLA eIBD and its holistic approach led to improved patient experience and satisfaction, which can provide useful recommendations for future electronic health solutions.

Personal Electronic Records of Medications (PERMs) for medication reconciliation at care transitions: a rapid realist review

Background

Medication reconciliation (MedRec), a process to reduce medication error at care transitions, is labour- and resource-intensive and time-consuming. Use of Personal Electronic Records of Medications (PERMs) in health information systems to support MedRec have proven challenging. Relatively little is known about the design, use or implementation of PERMs at care transitions that impacts on MedRec in the ‘real world’. To respond to this gap in knowledge we undertook a rapid realist review (RRR). The aim was to develop theories to explain how, why, when, where and for whom PERMs are designed, implemented or used in practice at care transitions that impacts on MedRec.

Methodology

We used realist methodology and undertook the RRR between August 2020 and February 2021. We collaborated with experts in the field to identify key themes. Articles were sourced from four databases (Pubmed, Embase, CINAHL Complete and OpenGrey) to contribute to the theory development. Quality assessment, screening and data extraction using NVivo was completed. Contexts, mechanisms and outcomes configurations were identified and synthesised. The experts considered these theories for relevance and practicality and suggested refinements.

Results

Ten provisional theories were identified from 19 articles. Some theories relate to the design (T2 Inclusive design, T3 PERMs complement existing good processes, T7 Interoperability), some relate to the implementation (T5 Tailored training, T9 Positive impact of legislation or governance), some relate to use (T6 Support and on-demand training) and others relate iteratively to all stages of the process (T1 Engage stakeholders, T4 Build trust, T8 Resource investment, T10 Patients as users of PERMs).

Conclusions

This RRR has allowed additional valuable data to be extracted from existing primary research, with minimal resources, that may impact positively on future developments in this area. The theories are interdependent to a greater or lesser extent; several or all of the theories may need to be in play to collectively impact on the design, implementation or use of PERMs for MedRec at care transitions. These theories should now be incorporated into an intervention and evaluated to further test their validity.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-021-01659-8.

Usability Evaluation of an Offline Electronic Data Capture App in a Prospective Multicenter Dementia Registry (digiDEM Bayern): Mixed Method Study

BACKGROUND

Digital registries have been shown to provide an efficient way of gaining a better understanding of the clinical complexity and long-term progression of diseases. The paperless method of electronic data capture (EDC) during a patient interview saves both time and resources. In the prospective multicenter project "Digital Dementia Registry Bavaria (digiDEM Bayern)," interviews are also performed on site in rural areas with unreliable internet connectivity. It must be ensured that EDC can still be performed in such a context and that there is no need to fall back on paper-based questionnaires. In addition to a web-based data collection solution, the EDC system REDCap (Research Electronic Data Capture) offers the option to collect data offline via an app and to synchronize it afterward.

OBJECTIVE

The aim of this study was to evaluate the usability of the REDCap app as an offline EDC option for a lay user group and to examine the necessary technology acceptance of using mobile devices for data collection. The feasibility of the app-based offline data collection in the digiDEM Bayern dementia registry project was then evaluated before going live.

METHODS

An exploratory mixed method design was employed in the form of an on-site usability test with the "Thinking Aloud" method combined with an online questionnaire including the System Usability Scale (SUS). The acceptance of mobile devices for data collection was surveyed based on five categories of the technology acceptance model.

RESULTS

Using the "Thinking Aloud" method, usability issues were identified and solutions were accordingly derived. Evaluation of the REDCap app resulted in a SUS score of 74, which represents "good" usability. After evaluating the technology acceptance questionnaire, it can be concluded that the lay user group is open to mobile devices as interview tools.

CONCLUSIONS

The usability evaluation results show that a lay user group generally agree that data collecting partners in the digiDEM project can handle the REDCap app well. The usability evaluation provided statements about positive aspects and could also identify usability issues relating to the REDCap app. In addition, the current technology acceptance in the sample showed that heterogeneous groups of different ages with diverse experiences in handling mobile devices are also ready for the use of app-based EDC systems. Based on these results, it can be assumed that the offline use of an app-based EDC system on mobile devices is a viable solution for collecting data in a decentralized registry-based research project.

Developing eHealth tools for diverse older adults: Lessons learned from the PREPARE for Your Care Program

BACKGROUND/OBJECTIVES

Electronic Health (eHealth) tools offer opportunities for people to access health information online; yet, most tools are not designed to meet the unique needs of diverse older adults, leading to health disparities. Our goal was to provide guidance for the development of eHealth tools for diverse older populations for use in geriatric care models.

DESIGN

Guidance for eHealth tools was compiled from user design resources and eHealth design literature. Pragmatic examples were provided from an evidenced-based eHealth tool called PREPAREforYourCare.org (PREPARE). We used quantitative feasibility data from PREPARE research studies and qualitative analysis of PREPARE focus groups, cognitive interviews, and feedback from randomized trials to further inform our recommendations.

RESULTS

Guidance and lessons learned include: (1) define clear objectives and a conceptual framework; (2) co-create with the target population; (3) optimize the design and layout for accessibility and ease of use, such as text at the 5th grade reading level, closed captioning, etc.; (4) use simple, standardized navigation design; (5) use actionable information to enhance behavior change, such as modeling of behaviors; (6) align accompanying written materials with the eHealth tool; and (7) create tracking mechanisms for ongoing user feedback. PREPARE is used as a case example to provide pragmatic illustrations for how the guidance may be operationalized.

CONCLUSION

eHealth tools can be tailored to the unique characteristics, preferences, and needs of diverse older populations. Following the "lessons learned" may help decrease health disparities among diverse older adults and ensure eHealth tools are readily accessible and culturally appropriate.

Usability Evaluation by Nurses of a Knowledge-Based System of Care Plans for People Hospitalized by COVID-19

The aim of this study is to analyze the usability by nurses of the Knowledge-Based System "Diagnostics care for COVID-19." A convenience sample of 16 nurses was selected, among hospital workers and external experts. The group was divided into three subgroups intentionally to obtain different usability perspectives. Usability was evaluated by the System Usability Scale questionnaire. The participants completed the questionnaire on general usability, data inputs, and information output, after completing a minimum of 12 care plans. The first subgroup used real cases and the "think aloud" technique, the second simulated cases from the same hospital, and the third subgroup performed the external simulation. The highest scores were obtained in data inputs (94.38-97.50); and the lowest, in general usability (90.00-95.00). The subgroup of external experts scored the highest (93.13-95.63), and the first subgroup, which carried out real cases, gave the lowest score (90.00-94.38). The "think aloud" technique found an improvement in including more diagnoses and being able to carry out several plans for one person at the same time. The usability obtained was "excellent" in all subgroups and questionnaires, although the application showed limitations related to its characteristics imposed in the requirements specification.

Development of and Experiences With an Informational Website on Early Labor: Qualitative User Involvement Study

Background

The period of regular contractions before 4 cm of cervical dilatation is often referred to as the latent phase or early labor. Women find it challenging to prepare for and cope with this phase of labor, and easily accessed web-based information from reliable sources may be useful in this preparation.

Objective

The aim of this study is to describe the development of a Norwegian website, Latens.no, for people seeking information on early labor and to explore users’ experiences with the website to increase its user-friendliness.

Methods

We developed a website using an iterative process involving a multidisciplinary research team, health personnel, users, a graphic designer, and an expert in software development. We explored the website’s user-friendliness using semistructured individual interviews and the think-aloud method. All interviews were audio recorded and transcribed. We then analyzed the participants’ feedback on the website.

Results

Participants included women who had recently given birth to their first baby (n=2), women who were pregnant with their first baby (n=4), and their partners (n=2). Results from participants’ experiences completing tasks included positive feedback related to the content of Latens.no, positive feedback related to the website’s design, and suggestions for improvement. Participants wanted to find information on early labor on the internet. Moreover, they found the information on the website relevant, trustworthy, and easy to read, and the design was attractive and easy to use. Overall, the participants performed the tasks easily, with few clicks and minimal effort.

Conclusions

The think-aloud method, while performing tasks, allowed for detailed feedback. The participants confirmed the user-friendliness of the website but at the same time provided information enabling improvement. We expect that changes made based on this user-centered design study will further increase the usability and acceptability of Latens.no.

Stakeholders' Perspectives on eHealth Support in Colorectal Cancer Survivorship: Qualitative Interview Study

Background

eHealth interventions may represent the way forward in following up patients with colorectal cancer (CRC) after hospital discharge to support them in coping with the illness, strengthen their self-management, and increase their quality of life. By involving end users of eHealth in cocreation processes when designing eHealth solutions, an acceptable and relevant product can be secured. Stakeholders’ perspectives could aid in closing the gap between research-developed products and the implementation of eHealth services in real-life scenarios.

Objective

This study aims to explore the views of patients with CRC, their informal caregivers, and health care professionals (HCPs) on information technology and the design of eHealth support in CRC care.

Methods

A qualitative, explorative design was used to conduct 31 semistructured individual interviews with 41% (13/31) patients with CRC, 29% (9/31) informal caregivers, and 29% (9/31) HCPs recruited from the gastrosurgical ward of a university hospital in southwestern Norway. A semistructured interview guide was used for data collection, and the data were analyzed by systematic text condensation.

Results

Participants described the diverse experiences of patients with CRC seeking web-based information. Age and digital competence were highlighted as influencers of the use of information technology. Patients rarely received advice from HCPs about relevant and secure websites containing information on CRC diagnosis and treatment. Features of desired eHealth interventions in following up patients with CRC were patient education, health monitoring, and communication with HCPs.

Conclusions

Several elements affect the activities of patients with CRC seeking health information. Age, inexperience with computer technology, and lack of access to web-based health information may reduce the ability of patients with CRC to engage in decision-making processes regarding illness and treatment. An eHealth service for patients with CRC should comprise features for information, education, and support for self-management and should aim to be individually adapted to the patient’s age and digital competence. Involving end users of eHealth services is necessary to ensure high-quality tailored services that are perceived as user friendly and relevant to the end users.

Democratizing epilepsy care: Utility and usability of an electronic patient portal

OBJECTIVES

Electronic patient portals (ePortals) can facilitate greater healthcare democratization by providing patients and/or their authorized care partners with secure access to their medical records when and where needed. Such democratization can promote effective healthcare provider-patient partnerships, shared decision-making, and greater patient engagement in managing their health condition. This study examined the usefulness of providing individualized services and care in epilepsy (PiSCES), an epilepsy ePortal, as an enabler of more democratized epilepsy care.

METHODS

Seventy-two individuals with epilepsy and 18 care partners were invited to report on their experience of interacting via PiSCES with clinical documents (epilepsy care summary record; epilepsy clinic letters) authored about them by healthcare providers. The OpenNotes reporting tool was adapted to capture participant experience.

RESULTS

Twenty-five percent of invited patients and 44% of invited care partners reported on interacting with their epilepsy care summary; 14% of patients and 67% of care partners invited reported on their epilepsy clinic letters. Participant testimonials illustrate the value of PiSCES in: promoting autonomy, aiding memory, developing the knowledgeable patient, and enhancing healthcare partnerships. Ninety-six percent and 100% of respondents, respectively, reported understanding their epilepsy care summary and epilepsy clinic letter; 77% said the summary described their epilepsy history to date; 96% indicated that the letter provided an accurate description of their clinical encounter; 92% and 96%, respectively, valued access to their summary record and clinic letters; 77% of summary record and 73% clinic letter respondents reported learning something about their epilepsy or the healthcare service via PiSCES. Illustrating their potential patient and care partner safety role, 42% respondents identified inaccuracies in their clinical documents which were subsequently resolved by a clinician.

SIGNIFICANCE

In the post-digital world highly customized on-demand products and services have come to be expected. Similarly, in epilepsy care, technologies such as PiSCES can enable more personalized, transparent, and engaging services.

eHealth in Geriatric Rehabilitation: Systematic Review of Effectiveness, Feasibility, and Usability

BACKGROUND

eHealth has the potential to improve outcomes such as physical activity or balance in older adults receiving geriatric rehabilitation. However, several challenges such as scarce evidence on effectiveness, feasibility, and usability hinder the successful implementation of eHealth in geriatric rehabilitation.

OBJECTIVE

The aim of this systematic review was to assess evidence on the effectiveness, feasibility, and usability of eHealth interventions in older adults in geriatric rehabilitation.

METHODS

We searched 7 databases for randomized controlled trials, nonrandomized studies, quantitative descriptive studies, qualitative research, and mixed methods studies that applied eHealth interventions during geriatric rehabilitation. Included studies investigated a combination of effectiveness, usability, and feasibility of eHealth in older patients who received geriatric rehabilitation, with a mean age of ≥70 years. Quality was assessed using the Mixed Methods Appraisal Tool and a narrative synthesis was conducted using a harvest plot.

RESULTS

In total, 40 studies were selected, with clinical heterogeneity across studies. Of 40 studies, 15 studies (38%) found eHealth was at least as effective as non-eHealth interventions (56% of the 27 studies with a control group), 11 studies (41%) found eHealth interventions were more effective than non-eHealth interventions, and 1 study (4%) reported beneficial outcomes in favor of the non-eHealth interventions. Of 17 studies, 16 (94%) concluded that eHealth was feasible. However, high exclusion rates were reported in 7 studies of 40 (18%). Of 40 studies, 4 (10%) included outcomes related to usability and indicated that there were certain aging-related barriers to cognitive ability, physical ability, or perception, which led to difficulties in using eHealth.

CONCLUSIONS

eHealth can potentially improve rehabilitation outcomes for older patients receiving geriatric rehabilitation. Simple eHealth interventions were more likely to be feasible for older patients receiving geriatric rehabilitation, especially, in combination with another non-eHealth intervention. However, a lack of evidence on usability might hamper the implementation of eHealth. eHealth applications in geriatric rehabilitation show promise, but more research is required, including research with a focus on usability and participation.

Implementation and Evaluation of the Premature Infant Pain Profile-revised (PIPP-R) e-Learning Module for Assessing Pain in Infants

OBJECTIVES

The Premature Infant Pain Profile-revised (PIPP-R) is a well-established measure for infant pain assessment. The aim of this study was to evaluate the implementation and clinical utility of the PIPP-R electronic learning (e-Learning) module to promote standardized health care training for nurses.

MATERIALS AND METHODS

A descriptive mixed-methods study was conducted in 2 tertiary Neonatal Intensive Care Units in Canada. Nurses were recruited and asked to complete the PIPP-R e-Learning Module and evaluate it. A 26-item questionnaire was used to describe nurse demographics and clinical experience and to evaluate implementation success (ie, acceptability, feasibility, usability) and clinical utility.

RESULTS

In all, 98 nurses from 2 settings in Central and Eastern Canada participated; most were registered nurses highly experienced in neonatal nursing care. The majority had received previous training on the PIPP-R (61.2%) and routinely used it in practice (67.4%). They considered the e-Learning module as acceptable and feasible as it was easy to access (94.9%) and to navigate (94.8%). Content was considered clear (98.9%) and met users' learning needs (99.0%). Nurses agreed that completing the module improved their understanding of neonatal pain (96.0%) and was clinically useful in improving their ability to assess pain in neonates (97.9%). The module was accessed primarily from work settings (77.8%) using desktop computers (49.0%) or tablets (28.0%) and was usually completed in a single session (75.7%).

DISCUSSION

Nurses' evaluation of the PIPP-R e-Learning module was overwhelmingly positive. The module was perceived as easy to implement, clinically useful, and was considered as a promising online educational tool. Further testing in clinical practice is needed to build on the results of this study and support the importance of dissemination of this module for standardized training purposes.

Usability Testing of a Mobile Health Application for Self-Management of Serious Mental Illness in a Norwegian Community Mental Health Setting

Background: For digital tools to have high usability and fit service users’ health needs and socio-environmental context, it is important to explore usability with end-users and identify facilitators and barriers to uptake. Objective: To conduct user testing of the smartphone health application, PeerTECH, in a Norwegian community mental health setting. Methods: Semistructured interviews and usability testing of the PeerTECH app using the Think-Aloud approach and task analysis among 11 people (three individuals with a serious mental illness, two peer support workers, and six mental health professionals). Results: Study participants perceived PeerTECH as a relevant tool to support self-management of their mental and physical health conditions, and they provided valuable feedback on existing features as well as suggestions for adaptions to the Norwegian context. The task analysis revealed that PeerTECH is easy to manage for service users and peer support workers. Conclusions: Adapting the PeerTECH smartphone app to the Norwegian context may be a viable and useful tool to support individuals with serious mental illness.