Methods of usability testing in the development of eHealth applications: A scoping review

A roadmap for the development and evaluation of the eHealthResp online course

Background

Inappropriate antibiotic use constitutes one of the most concerning public health issues, being one of the main causes of antibiotic resistance. Hence, to tackle this issue, it is important to encourage the development of educational interventions for health practitioners, namely by using digital health tools. This study focuses on the description of the development and validation process of the eHealthResp online course, a web platform directed to physicians and pharmacists, with the overall goal of improving antibiotic use for respiratory tract infections, along with the assessment of its usability.

Methods

The eHealthResp platform and the courses, developed with a user-centered design and based on Wordpress and MySQL, were based on a previously developed online course. A questionnaire to assess the usability was distributed among physicians (n = 6) and pharmacists (n = 6). Based on the obtained results, statistical analyses were conducted to calculate the usability score and appraise the design of the online course, as well as to compare the overall scores attributed by both groups. Further qualitative comments provided by the participants have also been analyzed.

Results

The eHealthResp contains two online courses directed to physicians and pharmacists aiming to aid in the management of respiratory tract infections. The average usability score of the eHealthResp online courses for physicians and pharmacists was of 78.33 (±11.57, 95%CI), and 83.75 (±15.90, 95%CI), respectively. Qualitative feedback emphasized the usefulness of the course, including overall positive reviews regarding user-friendliness and consistency.

Conclusions

This study led us to conclude that the eHealthResp online course is not recognized as a complex web platform, as both qualitative and quantitative feedback obtained were globally positive.

Adolescents’ opinions on the use of a smartphone application as an oral health education tool: A qualitative study

Developing health-related smartphone applications for oral health education should consider the end-user's perspectives to ensure they will be usable. This study aimed to explore the opinions of secondary school students in Selangor, Malaysia regarding the use of a smartphone application for oral health education and to identify the features for an oral health education smartphone application from the perspectives of adolescents. Focus group discussions were conducted among Form Two (14-year-old) and Form Four (16-year-old) students from selected government secondary schools in Selangor utilising a semi-structured topic guide until data saturation was achieved. Data were analysed using framework analysis. Ten focus group discussions were conducted involving 77 participants. Mixed opinions were recorded on the use of health-related smartphone applications for oral health education. The preferred features in a health-related smartphone application are disease detection, have games and rewards, educational and fun, access to a dentist, reminders, and user-friendliness. Adolescents are aware of the positive aspect of using health-related smartphone applications for oral health education; however, they are wary of the need to install one. Nevertheless, identifying adolescents’ preferred features of an oral health education app is the first step in developing an application tailored to their needs. Smartphone application could be a timely strategy to improve oral health education delivery and behaviour improvement for this age group.

Enhancing System Acceptance through User-Centred Design: Integrating Patient Generated Wellness Data

Gestational diabetes mellitus (GDM) is a condition that appears during pregnancy and is expected to be a temporary one. While patients are encouraged to manage it themselves, research findings indicate that GDM may negatively affect the foetus; in addition, there is an increased risk of women with GDM subsequently developing Type 2 diabetes. To alleviate the risks, women with GDM are advised to maintain a record of their diet and blood glucose levels and to attend regular clinical reviews. Rather than using a paper diary, women with GDM can maintain a record of their blood glucose level readings and other relevant data using a wellness mobile application (app). However, such apps are developed for general use and may not meet the specific needs of clinical staff (physicians, dietitians, obstetricians and midwives) involved in managing GDM; for example, an app may record glucose readings but not the details of a meal taken before or after the glucose reading. Second, the apps do not permanently store the data generated by the patient and do not support the transfer of these data to a clinical system or information portal. The mobile health (mHealth) system designed and developed in this research allows one to integrate different types of user generated wellness data into a centralised database. A user-centered design (UCD) approach informed by the technology acceptance model (TAM) was adopted. This paper investigates and evaluates the effectiveness of the approach with regard to facilitating system acceptance and future adoption through an early focus on enhancing system usefulness and ease of use. The functional system requirements of the proposed system were refined through a series of interviews with the perspective of clinical users; ease-of-use and usability issues were resolved through 'think aloud' sessions with clinicians and GDM patients.

Online self-exclusion from multiple gambling venues: Stakeholder co-design of a usable and acceptable self-directed website

Gambling self-exclusion programs are under-utilised and barriers to entry include shame and embarrassment with face-to-face registration, and complex and effortful procedures. The current study aimed to facilitate self-exclusion from gambling venues via an online self-directed website. A co-design approach was used to elicit key stakeholders' perspectives on required website features, functionality, and to identify variables potentially impacting on development and implementation. Semi-structured focus groups and interviews were conducted across four stakeholders (N = 25): self-exclusion end users (consumers, n = 5), gambling counsellors (n = 7), venue staff (n = 6), and policy makers (n = 7). Overall, stakeholder perspectives were consistent with content analysis indicating the importance of website user-friendliness, flexibility, supportiveness, and trustworthiness. Importantly, these attributes were linked to target end users': perceived vulnerabilities, diverse backgrounds and individual expectations. Participants believed that the entire self-exclusion process should be conducted online, including identity verification, whilst expecting high-level data security measures to protect their personal privacy. A separate webpage was also suggested containing relevant information and links to additional help services, such as counselling. This study describes an adaptable co-design framework for developing a usable and acceptable self-exclusion website. Future studies should empirically test system usability and acceptability to refine and maximise system uptake upon implementation. Findings may have broader implications for digital health intervention design.

Upper limb rehabilitation system based on virtual reality for breast cancer patients: Development and usability study

Background

Functional exercise is crucial for breast cancer patients after surgery, and the use of virtual reality technology to assist patients with postoperative upper limb functional rehabilitation has gradually attracted the attention of researchers. However, the usability of the developed rehabilitation system is still unknown to a large extent. The purpose of this study was to develop a virtual reality upper limb rehabilitation system for patients after breast cancer surgery and to explore its usability.

Methods

We built a multidisciplinary team based on virtual reality and human-computer interaction technology and designed and developed an upper limb function rehabilitation system for breast cancer patients after surgery. Breast cancer patients were recruited from a grade III-a general hospital in Changchun city for the experiment. We used the System Usability Scale to evaluate the system availability, the Presence Questionnaire scale to measure the immersive virtual reality scene, and the Simulator Sickness Questionnaire subjective measurement scale for simulator sickness symptoms.

Results

This upper limb rehabilitation system hardware consisted of Head-mounted Display, a control handle and notebook computers. The software consisted of rehabilitation exercises and game modules. A total of 15 patients were tested on this system, all of whom were female. The mean age was 54.73±7.78 years, and no patients were excluded from the experiment because of adverse reactions such as dizziness and vomiting. The System Usability Scale score was 90.50±5.69, the Presence Questionnaire score was 113.40±9.58, the Simulator Sickness Questionnaire-nausea score was 0.93±1.16, the Simulator Sickness Questionnaire-oculomotor score was 0.80±1.27, the Simulator Sickness Questionnaire-disorientation score was 0.80±1.27, and the Simulator Sickness Questionnaire total score was 2.53±3.40.

Conclusions

This study fills in the blanks regarding the upper limb rehabilitation of breast cancer patients based on virtual reality technology system usability research. As the starting point of research in the future, we will improve the system’s function and design strictly randomized controlled trials, using larger samples in the promotion, to evaluate its application in breast cancer patients with upper limbs and other physiological functions and the feasibility and effects of rehabilitation.

An Intelligent Individualized Cardiovascular App for Risk Elimination (iCARE) for Individuals With Coronary Heart Disease: Development and Usability Testing Analysis

Background

Death and disability from coronary heart disease (CHD) can be largely reduced by improving risk factor management. However, adhering to evidence-based recommendations is challenging and requires interventions at the level of the patient, provider, and health system.

Objective

The aim of this study was to develop an Intelligent Individualized Cardiovascular App for Risk Elimination (iCARE) to facilitate adherence to health behaviors and preventive medications, and to test the usability of iCARE.

Methods

We developed iCARE based on a user-centered design approach, which included 4 phases: (1) function design, (2) iterative design, (3) expert inspections and walkthroughs of the prototypes, and (4) usability testing with end users. The usability testing of iCARE included 2 stages: stage I, which included a task analysis and a usability evaluation (January to March 2019) of the iCARE patient app using the modified Health Information Technology Usability Survey (Health-ITUES); and stage II (June 2020), which used the Health-ITUES among end users who used the app for 6 months. The end users were individuals with a confirmed diagnosis of CHD from 2 university-affiliated hospitals in Beijing, China.

Results

iCARE consists of a patient app, a care provider app, and a cloud platform. It has a set of algorithms that trigger tailored feedback and can send individualized interventions based on data from initial assessment and health monitoring via manual entry or wearable devices. For stage I usability testing, 88 hospitalized patients (72% [63/88] male; mean age 60 [SD 9.9] years) with CHD were included in the study. The mean score of the usability testing was 90.1 (interquartile range 83.3-99.0). Among enrolled participants, 90% (79/88) were satisfied with iCARE; 94% (83/88) and 82% (72/88) reported that iCARE was useful and easy to use, respectively. For stage II usability testing, 61 individuals with CHD (85% [52/61] male; mean age 53 [SD 8.2] years) who were from an intervention arm and used iCARE for at least six months were included. The mean total score on usability testing based on the questionnaire was 89.0 (interquartile distance: 77.0-99.5). Among enrolled participants, 89% (54/61) were satisfied with the use of iCARE, 93% (57/61) perceived it as useful, and 70% (43/61) as easy to use.

Conclusions

This study developed an intelligent, individualized, evidence-based, and theory-driven app (iCARE) to improve patients’ adherence to health behaviors and medication management. iCARE was identified to be highly acceptable, useful, and easy to use among individuals with a diagnosis of CHD.

Trial Registration

Chinese Clinical Trial Registry ChiCTR-INR-16010242; https://tinyurl.com/2p8bkrew

Exploring the relationship between the usability of a goal-oriented mobile health application and non-usage attrition in patients with multimorbidity: A blended data analysis approach

Background

Mobile health applications are increasingly used to support the delivery of health care services to a variety of patients. Based on data obtained from a pragmatic trial of the electronic Patient Reported Outcome (ePRO) app designed to support goal-oriented care primary care, this study aims to (1) examine how patient-reported usability changed over the one-year intervention period, and (2) explore participant attrition rate of the electronic Patient Reported Outcome app over one year study period.

Methods

We performed a secondary analysis of 44 older adults with complex chronic needs enrolled in the electronic Patient Reported Outcome-digital health intervention. App usage and attrition were measured using device-generated usage logs; usability was measured using the patient-reported post-study system usability questionnaire collected at 3, 6, 9, and 12 months. Research memos were used to interpret potential contextual contributing factors to patients' overall usage and usability score pattern. A data triangulation method of both quantitative and qualitative data was used to analyze and interpret study findings.

Results

While there was gradual attrition in the use of the ePRO app, patients' usability scores remained consistent throughout the study period. Qualitative memos suggested patients' encounters with technical difficulties and relationship dynamics with primary providers influenced patients' adherence to the ePRO app.

Conclusion

This study highlights that the patient-provider relationship is a key determining factor that influences complex patients' continued engagement with a Mobile health app. The finding calls attention to the measurement of usability of a Mobile health app, its impact on attrition, and contributing factors that influence patients' attrition. Trial registration: Clinicaltrials.gov Identified NCT02917954.

Patient Experience and Satisfaction with an e-Health Care Management Application for Inflammatory Bowel Diseases

Background: Rising healthcare expenditures have been partially attributed to suboptimal management of inflammatory bowel diseases (IBD). Electronic health interventions may help improve care management for IBD patients, but there is a need to better understand patient perspectives on these emerging technologies. Aims: The primary aim was to evaluate patient satisfaction and experience with the UCLA eIBD mobile application, an integrative care management platform with disease activity monitoring tools and educational modules. The secondary objective was to capture patient feedback on how to improve the mobile application. Methods: We surveyed IBD patients treated at the UCLA Center for Inflammatory Bowel Diseases. The patient experience survey assessed the patients’ overall satisfaction with the application, perception of health outcomes after participation in the program, and feedback on educational modules as well as areas for application improvement. Results: 50 patients were included. The responses indicated that the patients were greatly satisfied with the ease of patient–provider communication within the application and appointment scheduling features (68%). A majority of respondents (54%) also reported that program participation resulted in improved perception of disease control and quality of life. Lastly, a majority of participants (79%) would recommend this application to others. Conclusions: Mobile tools such as UCLA eIBD have promising implications for integration into patients’ daily lives. This patient satisfaction study suggests the feasibility of using this mobile application by patients and providers. We further showed that UCLA eIBD and its holistic approach led to improved patient experience and satisfaction, which can provide useful recommendations for future electronic health solutions.

Personal Electronic Records of Medications (PERMs) for medication reconciliation at care transitions: a rapid realist review

Background

Medication reconciliation (MedRec), a process to reduce medication error at care transitions, is labour- and resource-intensive and time-consuming. Use of Personal Electronic Records of Medications (PERMs) in health information systems to support MedRec have proven challenging. Relatively little is known about the design, use or implementation of PERMs at care transitions that impacts on MedRec in the ‘real world’. To respond to this gap in knowledge we undertook a rapid realist review (RRR). The aim was to develop theories to explain how, why, when, where and for whom PERMs are designed, implemented or used in practice at care transitions that impacts on MedRec.

Methodology

We used realist methodology and undertook the RRR between August 2020 and February 2021. We collaborated with experts in the field to identify key themes. Articles were sourced from four databases (Pubmed, Embase, CINAHL Complete and OpenGrey) to contribute to the theory development. Quality assessment, screening and data extraction using NVivo was completed. Contexts, mechanisms and outcomes configurations were identified and synthesised. The experts considered these theories for relevance and practicality and suggested refinements.

Results

Ten provisional theories were identified from 19 articles. Some theories relate to the design (T2 Inclusive design, T3 PERMs complement existing good processes, T7 Interoperability), some relate to the implementation (T5 Tailored training, T9 Positive impact of legislation or governance), some relate to use (T6 Support and on-demand training) and others relate iteratively to all stages of the process (T1 Engage stakeholders, T4 Build trust, T8 Resource investment, T10 Patients as users of PERMs).

Conclusions

This RRR has allowed additional valuable data to be extracted from existing primary research, with minimal resources, that may impact positively on future developments in this area. The theories are interdependent to a greater or lesser extent; several or all of the theories may need to be in play to collectively impact on the design, implementation or use of PERMs for MedRec at care transitions. These theories should now be incorporated into an intervention and evaluated to further test their validity.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-021-01659-8.

Usability Evaluation of an Offline Electronic Data Capture App in a Prospective Multicenter Dementia Registry (digiDEM Bayern): Mixed Method Study

BACKGROUND

Digital registries have been shown to provide an efficient way of gaining a better understanding of the clinical complexity and long-term progression of diseases. The paperless method of electronic data capture (EDC) during a patient interview saves both time and resources. In the prospective multicenter project "Digital Dementia Registry Bavaria (digiDEM Bayern)," interviews are also performed on site in rural areas with unreliable internet connectivity. It must be ensured that EDC can still be performed in such a context and that there is no need to fall back on paper-based questionnaires. In addition to a web-based data collection solution, the EDC system REDCap (Research Electronic Data Capture) offers the option to collect data offline via an app and to synchronize it afterward.

OBJECTIVE

The aim of this study was to evaluate the usability of the REDCap app as an offline EDC option for a lay user group and to examine the necessary technology acceptance of using mobile devices for data collection. The feasibility of the app-based offline data collection in the digiDEM Bayern dementia registry project was then evaluated before going live.

METHODS

An exploratory mixed method design was employed in the form of an on-site usability test with the "Thinking Aloud" method combined with an online questionnaire including the System Usability Scale (SUS). The acceptance of mobile devices for data collection was surveyed based on five categories of the technology acceptance model.

RESULTS

Using the "Thinking Aloud" method, usability issues were identified and solutions were accordingly derived. Evaluation of the REDCap app resulted in a SUS score of 74, which represents "good" usability. After evaluating the technology acceptance questionnaire, it can be concluded that the lay user group is open to mobile devices as interview tools.

CONCLUSIONS

The usability evaluation results show that a lay user group generally agree that data collecting partners in the digiDEM project can handle the REDCap app well. The usability evaluation provided statements about positive aspects and could also identify usability issues relating to the REDCap app. In addition, the current technology acceptance in the sample showed that heterogeneous groups of different ages with diverse experiences in handling mobile devices are also ready for the use of app-based EDC systems. Based on these results, it can be assumed that the offline use of an app-based EDC system on mobile devices is a viable solution for collecting data in a decentralized registry-based research project.

Developing eHealth tools for diverse older adults: Lessons learned from the PREPARE for Your Care Program

BACKGROUND/OBJECTIVES

Electronic Health (eHealth) tools offer opportunities for people to access health information online; yet, most tools are not designed to meet the unique needs of diverse older adults, leading to health disparities. Our goal was to provide guidance for the development of eHealth tools for diverse older populations for use in geriatric care models.

DESIGN

Guidance for eHealth tools was compiled from user design resources and eHealth design literature. Pragmatic examples were provided from an evidenced-based eHealth tool called PREPAREforYourCare.org (PREPARE). We used quantitative feasibility data from PREPARE research studies and qualitative analysis of PREPARE focus groups, cognitive interviews, and feedback from randomized trials to further inform our recommendations.

RESULTS

Guidance and lessons learned include: (1) define clear objectives and a conceptual framework; (2) co-create with the target population; (3) optimize the design and layout for accessibility and ease of use, such as text at the 5th grade reading level, closed captioning, etc.; (4) use simple, standardized navigation design; (5) use actionable information to enhance behavior change, such as modeling of behaviors; (6) align accompanying written materials with the eHealth tool; and (7) create tracking mechanisms for ongoing user feedback. PREPARE is used as a case example to provide pragmatic illustrations for how the guidance may be operationalized.

CONCLUSION

eHealth tools can be tailored to the unique characteristics, preferences, and needs of diverse older populations. Following the "lessons learned" may help decrease health disparities among diverse older adults and ensure eHealth tools are readily accessible and culturally appropriate.

Usability Evaluation by Nurses of a Knowledge-Based System of Care Plans for People Hospitalized by COVID-19

The aim of this study is to analyze the usability by nurses of the Knowledge-Based System "Diagnostics care for COVID-19." A convenience sample of 16 nurses was selected, among hospital workers and external experts. The group was divided into three subgroups intentionally to obtain different usability perspectives. Usability was evaluated by the System Usability Scale questionnaire. The participants completed the questionnaire on general usability, data inputs, and information output, after completing a minimum of 12 care plans. The first subgroup used real cases and the "think aloud" technique, the second simulated cases from the same hospital, and the third subgroup performed the external simulation. The highest scores were obtained in data inputs (94.38-97.50); and the lowest, in general usability (90.00-95.00). The subgroup of external experts scored the highest (93.13-95.63), and the first subgroup, which carried out real cases, gave the lowest score (90.00-94.38). The "think aloud" technique found an improvement in including more diagnoses and being able to carry out several plans for one person at the same time. The usability obtained was "excellent" in all subgroups and questionnaires, although the application showed limitations related to its characteristics imposed in the requirements specification.

Development of and Experiences With an Informational Website on Early Labor: Qualitative User Involvement Study

Background

The period of regular contractions before 4 cm of cervical dilatation is often referred to as the latent phase or early labor. Women find it challenging to prepare for and cope with this phase of labor, and easily accessed web-based information from reliable sources may be useful in this preparation.

Objective

The aim of this study is to describe the development of a Norwegian website, Latens.no, for people seeking information on early labor and to explore users’ experiences with the website to increase its user-friendliness.

Methods

We developed a website using an iterative process involving a multidisciplinary research team, health personnel, users, a graphic designer, and an expert in software development. We explored the website’s user-friendliness using semistructured individual interviews and the think-aloud method. All interviews were audio recorded and transcribed. We then analyzed the participants’ feedback on the website.

Results

Participants included women who had recently given birth to their first baby (n=2), women who were pregnant with their first baby (n=4), and their partners (n=2). Results from participants’ experiences completing tasks included positive feedback related to the content of Latens.no, positive feedback related to the website’s design, and suggestions for improvement. Participants wanted to find information on early labor on the internet. Moreover, they found the information on the website relevant, trustworthy, and easy to read, and the design was attractive and easy to use. Overall, the participants performed the tasks easily, with few clicks and minimal effort.

Conclusions

The think-aloud method, while performing tasks, allowed for detailed feedback. The participants confirmed the user-friendliness of the website but at the same time provided information enabling improvement. We expect that changes made based on this user-centered design study will further increase the usability and acceptability of Latens.no.

Stakeholders' Perspectives on eHealth Support in Colorectal Cancer Survivorship: Qualitative Interview Study

Background

eHealth interventions may represent the way forward in following up patients with colorectal cancer (CRC) after hospital discharge to support them in coping with the illness, strengthen their self-management, and increase their quality of life. By involving end users of eHealth in cocreation processes when designing eHealth solutions, an acceptable and relevant product can be secured. Stakeholders’ perspectives could aid in closing the gap between research-developed products and the implementation of eHealth services in real-life scenarios.

Objective

This study aims to explore the views of patients with CRC, their informal caregivers, and health care professionals (HCPs) on information technology and the design of eHealth support in CRC care.

Methods

A qualitative, explorative design was used to conduct 31 semistructured individual interviews with 41% (13/31) patients with CRC, 29% (9/31) informal caregivers, and 29% (9/31) HCPs recruited from the gastrosurgical ward of a university hospital in southwestern Norway. A semistructured interview guide was used for data collection, and the data were analyzed by systematic text condensation.

Results

Participants described the diverse experiences of patients with CRC seeking web-based information. Age and digital competence were highlighted as influencers of the use of information technology. Patients rarely received advice from HCPs about relevant and secure websites containing information on CRC diagnosis and treatment. Features of desired eHealth interventions in following up patients with CRC were patient education, health monitoring, and communication with HCPs.

Conclusions

Several elements affect the activities of patients with CRC seeking health information. Age, inexperience with computer technology, and lack of access to web-based health information may reduce the ability of patients with CRC to engage in decision-making processes regarding illness and treatment. An eHealth service for patients with CRC should comprise features for information, education, and support for self-management and should aim to be individually adapted to the patient’s age and digital competence. Involving end users of eHealth services is necessary to ensure high-quality tailored services that are perceived as user friendly and relevant to the end users.

Democratizing epilepsy care: Utility and usability of an electronic patient portal

OBJECTIVES

Electronic patient portals (ePortals) can facilitate greater healthcare democratization by providing patients and/or their authorized care partners with secure access to their medical records when and where needed. Such democratization can promote effective healthcare provider-patient partnerships, shared decision-making, and greater patient engagement in managing their health condition. This study examined the usefulness of providing individualized services and care in epilepsy (PiSCES), an epilepsy ePortal, as an enabler of more democratized epilepsy care.

METHODS

Seventy-two individuals with epilepsy and 18 care partners were invited to report on their experience of interacting via PiSCES with clinical documents (epilepsy care summary record; epilepsy clinic letters) authored about them by healthcare providers. The OpenNotes reporting tool was adapted to capture participant experience.

RESULTS

Twenty-five percent of invited patients and 44% of invited care partners reported on interacting with their epilepsy care summary; 14% of patients and 67% of care partners invited reported on their epilepsy clinic letters. Participant testimonials illustrate the value of PiSCES in: promoting autonomy, aiding memory, developing the knowledgeable patient, and enhancing healthcare partnerships. Ninety-six percent and 100% of respondents, respectively, reported understanding their epilepsy care summary and epilepsy clinic letter; 77% said the summary described their epilepsy history to date; 96% indicated that the letter provided an accurate description of their clinical encounter; 92% and 96%, respectively, valued access to their summary record and clinic letters; 77% of summary record and 73% clinic letter respondents reported learning something about their epilepsy or the healthcare service via PiSCES. Illustrating their potential patient and care partner safety role, 42% respondents identified inaccuracies in their clinical documents which were subsequently resolved by a clinician.

SIGNIFICANCE

In the post-digital world highly customized on-demand products and services have come to be expected. Similarly, in epilepsy care, technologies such as PiSCES can enable more personalized, transparent, and engaging services.

eHealth in Geriatric Rehabilitation: Systematic Review of Effectiveness, Feasibility, and Usability

BACKGROUND

eHealth has the potential to improve outcomes such as physical activity or balance in older adults receiving geriatric rehabilitation. However, several challenges such as scarce evidence on effectiveness, feasibility, and usability hinder the successful implementation of eHealth in geriatric rehabilitation.

OBJECTIVE

The aim of this systematic review was to assess evidence on the effectiveness, feasibility, and usability of eHealth interventions in older adults in geriatric rehabilitation.

METHODS

We searched 7 databases for randomized controlled trials, nonrandomized studies, quantitative descriptive studies, qualitative research, and mixed methods studies that applied eHealth interventions during geriatric rehabilitation. Included studies investigated a combination of effectiveness, usability, and feasibility of eHealth in older patients who received geriatric rehabilitation, with a mean age of ≥70 years. Quality was assessed using the Mixed Methods Appraisal Tool and a narrative synthesis was conducted using a harvest plot.

RESULTS

In total, 40 studies were selected, with clinical heterogeneity across studies. Of 40 studies, 15 studies (38%) found eHealth was at least as effective as non-eHealth interventions (56% of the 27 studies with a control group), 11 studies (41%) found eHealth interventions were more effective than non-eHealth interventions, and 1 study (4%) reported beneficial outcomes in favor of the non-eHealth interventions. Of 17 studies, 16 (94%) concluded that eHealth was feasible. However, high exclusion rates were reported in 7 studies of 40 (18%). Of 40 studies, 4 (10%) included outcomes related to usability and indicated that there were certain aging-related barriers to cognitive ability, physical ability, or perception, which led to difficulties in using eHealth.

CONCLUSIONS

eHealth can potentially improve rehabilitation outcomes for older patients receiving geriatric rehabilitation. Simple eHealth interventions were more likely to be feasible for older patients receiving geriatric rehabilitation, especially, in combination with another non-eHealth intervention. However, a lack of evidence on usability might hamper the implementation of eHealth. eHealth applications in geriatric rehabilitation show promise, but more research is required, including research with a focus on usability and participation.

Implementation and Evaluation of the Premature Infant Pain Profile-revised (PIPP-R) e-Learning Module for Assessing Pain in Infants

OBJECTIVES

The Premature Infant Pain Profile-revised (PIPP-R) is a well-established measure for infant pain assessment. The aim of this study was to evaluate the implementation and clinical utility of the PIPP-R electronic learning (e-Learning) module to promote standardized health care training for nurses.

MATERIALS AND METHODS

A descriptive mixed-methods study was conducted in 2 tertiary Neonatal Intensive Care Units in Canada. Nurses were recruited and asked to complete the PIPP-R e-Learning Module and evaluate it. A 26-item questionnaire was used to describe nurse demographics and clinical experience and to evaluate implementation success (ie, acceptability, feasibility, usability) and clinical utility.

RESULTS

In all, 98 nurses from 2 settings in Central and Eastern Canada participated; most were registered nurses highly experienced in neonatal nursing care. The majority had received previous training on the PIPP-R (61.2%) and routinely used it in practice (67.4%). They considered the e-Learning module as acceptable and feasible as it was easy to access (94.9%) and to navigate (94.8%). Content was considered clear (98.9%) and met users' learning needs (99.0%). Nurses agreed that completing the module improved their understanding of neonatal pain (96.0%) and was clinically useful in improving their ability to assess pain in neonates (97.9%). The module was accessed primarily from work settings (77.8%) using desktop computers (49.0%) or tablets (28.0%) and was usually completed in a single session (75.7%).

DISCUSSION

Nurses' evaluation of the PIPP-R e-Learning module was overwhelmingly positive. The module was perceived as easy to implement, clinically useful, and was considered as a promising online educational tool. Further testing in clinical practice is needed to build on the results of this study and support the importance of dissemination of this module for standardized training purposes.

Usability Testing of a Mobile Health Application for Self-Management of Serious Mental Illness in a Norwegian Community Mental Health Setting

Background: For digital tools to have high usability and fit service users’ health needs and socio-environmental context, it is important to explore usability with end-users and identify facilitators and barriers to uptake. Objective: To conduct user testing of the smartphone health application, PeerTECH, in a Norwegian community mental health setting. Methods: Semistructured interviews and usability testing of the PeerTECH app using the Think-Aloud approach and task analysis among 11 people (three individuals with a serious mental illness, two peer support workers, and six mental health professionals). Results: Study participants perceived PeerTECH as a relevant tool to support self-management of their mental and physical health conditions, and they provided valuable feedback on existing features as well as suggestions for adaptions to the Norwegian context. The task analysis revealed that PeerTECH is easy to manage for service users and peer support workers. Conclusions: Adapting the PeerTECH smartphone app to the Norwegian context may be a viable and useful tool to support individuals with serious mental illness.

Mobile Health for Smoking Cessation Among Disadvantaged Young Women During and After Pregnancy: User-Centered Design and Usability Study

Background

Smoking prevalence during and after pregnancy remains high among socioeconomically disadvantaged women. Mobile health (mHealth) apps with game and social support elements seem promising to support smoking cessation.

Objective

This study aims to describe the user-centered design and usability evaluation of Kindle, an mHealth app with game and social support elements, to support disadvantaged young women during and after pregnancy through the first stages of smoking cessation.

Methods

Disadvantaged women (n=9), members of their social networks (n=4), and nurses supporting these women (n=51) were informants throughout the iterative prototype development of Kindle according to the International Organization for Standardization 9241-11:2018. Specific phases included understanding the context of use through secondary analysis of qualitative interview data (phase 1), establishing the user and organizational requirements (phase 2), production of design solutions (phase 3), and usability inspection of the prototype through a heuristic evaluation (3 experts) along with user testing by a think aloud method (5 disadvantaged women and 5 nurses; phase 4). Usability problems were categorized according to the principles of the Healthcare Information and Management Systems Society.

Results

Phase 1 resulted in an understanding of the VoorZorg program and the needs of VoorZorg nurses and clients (eg, focus on early stages of change and building new supportive networks to aid clients in smoking cessation). In phase 2, we established requirements (n=22; eg, mHealth app, secure communication between nurses and clients, easy-to-use interfaces, inclusion of game elements, and tailoring at early stages of change in smoking cessation). Phase 3 resulted in a prototype of Kindle, combining the interface for nurses and clients, including the following functionalities: personal goal setting with earning points; secured chat function between nurses and other clients; and tips, diary, and profile creation. The heuristic evaluation and thinking aloud method in phase 4 revealed 78 usability problems in the interfaces. Most usability problems concerned simplicity (eg, unclear clickable button) and naturalness (eg, unclear icon).

Conclusions

The user-centered design and usability testing of the mHealth app Kindle yielded useful insights. The involvement of end users, specifically socioeconomically disadvantaged women during and after their pregnancy, resulted in a prototype that met their needs and requirements (eg, mHealth app, secure communication between nurses and clients, easy-to-use interfaces, inclusion of game elements, and tailoring to the early stages of change in smoking cessation) to achieve readiness for smoking cessation. Moreover, the usability evaluation by end users and experts revealed unique usability problems for this population. These insights allow for further optimization of Kindle and encourage future studies to engage disadvantaged populations in all phases of mHealth intervention design and usability testing.

Voicing Individual Concerns for Engagement in Hemodialysis (VOICE-HD): A Mixed Method, Randomized Pilot Trial of Digital Health in Dialysis Care Delivery

BACKGROUND

People receiving in-center hemodialysis (HD) have prioritized the need for more individualized health information and better communication with nephrologists. The most common setting for patient-nephrologist interactions is during the HD treatment, which is a time pressured setting that lacks privacy.

OBJECTIVE

To facilitate effective communication in the hemodialysis (HD) unit, we evaluated the usability of a web application (web app) from both the patient and physician perspective. The main aim of the web app was to support patients in prioritizing their dialysis concerns outside of the clinical HD encounter.

DESIGN

Mixed method, parallel arm, multi-site, pilot randomized controlled trial.

SETTING

Two outpatient Canadian HD centers.

PARTICIPANTS

Adult patients receiving in-center HD and their attending nephrologists.

METHODS

Patients were randomized to either a web application or an active control (paper form) for logging concerns to be addressed at weekly encounters with the nephrologist over 8 weeks. Topics included: HD treatment, symptoms, modality, and medications. The primary outcome was usability, defined as effectiveness (engagement with the tool, frequency of submitted concerns, whether the concern was satisfactorily addressed) and satisfaction with the tool using a priori thresholds and explored in interviews with patients and nephrologists.

RESULTS

77 patients (30 women, median age 61, interquartile range [53,67], median 2 years [1,4] on dialysis) and 19 nephrologists (4 women, median age 46 [36,65]) were enrolled. Patient use of a digital device at baseline was low (20%). Engagement with the tool was 70% (web app) and 100% (paper) with a lower proportion of patients in the web app group submitting at least one concern over 8 weeks compared to the paper form group: 56.7% vs 87.9%. Weekly concerns were satisfactorily addressed in both groups and ≥70% of patients would continue to use the tools. For patients, both tools promoted preparation and participation in the encounter; however, only the web app facilitated greater privacy in relaying concerns. For most nephrologists, the tools were disruptive to their workflow and were perceived as unnecessary given existing processes and familiarity with patients. For future versions of the app, patients suggested more features to facilitate self-management and nephrologists suggested integration with health databases and multidisciplinary teams.

LIMITATIONS

Tertiary setting may limit generalizability.

CONCLUSIONS

Both tools promoted fundamental components of self-management; however, patients in the paper form group submitted concerns more often and this tool was easier to remember to use. Although modifications would likely enhance web app usability, successful future adoption is limited by physician acceptance.Trial registration ClinicalTrials.gov NCT03605875.

Conceptualizing Usability for the eHealth Context: Content Analysis of Usability Problems of eHealth Applications

BACKGROUND

Usability tests can be either formative (where the aim is to detect usability problems) or summative (where the aim is to benchmark usability). There are ample formative methods that consider user characteristics and contexts (ie, cognitive walkthroughs, interviews, and verbal protocols). This is especially valuable for eHealth applications, as health conditions can influence user-system interactions. However, most summative usability tests do not consider eHealth-specific factors that could potentially affect the usability of a system. One of the reasons for this is the lack of fine-grained frameworks or models of usability factors that are unique to the eHealth domain.

OBJECTIVE

In this study, we aim to develop an ontology of usability problems, specifically for eHealth applications, with patients as primary end users.

METHODS

We analyzed 8 data sets containing the results of 8 formative usability tests for eHealth applications. These data sets contained 400 usability problems that could be used for analysis. Both inductive and deductive coding were used to create an ontology from 6 data sets, and 2 data sets were used to validate the framework by assessing the intercoder agreement.

RESULTS

We identified 8 main categories of usability factors, including basic system performance, task-technology fit, accessibility, interface design, navigation and structure, information and terminology, guidance and support, and satisfaction. These 8 categories contained a total of 21 factors: 14 general usability factors and 7 eHealth-specific factors. Cohen κ was calculated for 2 data sets on both the category and factor levels, and all Cohen κ values were between 0.62 and 0.67, which is acceptable. Descriptive analysis revealed that approximately 69.5% (278/400) of the usability problems can be considered as general usability factors and 30.5% (122/400) as eHealth-specific usability factors.

CONCLUSIONS

Our ontology provides a detailed overview of the usability factors for eHealth applications. Current usability benchmarking instruments include only a subset of the factors that emerged from our study and are therefore not fully suited for summative evaluations of eHealth applications. Our findings support the development of new usability benchmarking tools for the eHealth domain.

The Danish Future Patient Telerehabilitation Program for Patients With Atrial Fibrillation: Design and Pilot Study in Collaboration With Patients and Their Spouses

Background

Atrial fibrillation (AF) is the most common cardiac arrhythmia and is predicted to more than double in prevalence over the next 20 years. Tailored patient education is recommended as an important aspect of AF care. Current guidelines emphasize that patients become more active participants in the management of their own disease, yet there are no rehabilitation programs for patients with AF in the Danish health care system. Through participatory design, we developed the Future Patient Telerehabilitation (TR) Programs, A and B, for patients with AF. The 2 programs are based on HeartPortal and remote monitoring, together with educational modules.

Objective

The aim of this pilot study is to evaluate and compare the feasibility of the 2 programs of TR for patients with AF.

Methods

This pilot study was conducted between December 2019 and March 2020. The pilot study consisted of testing the 2 TR programs, A and B, in two phases: (1) treatment at the AF clinic and (2) TR at home. The primary outcome of the study was the usability of technologies for self-monitoring and the context of the TR programs as seen from patients’ perspectives. Secondary outcomes were the development of patients’ knowledge of AF, development of clinical data, and understanding the expectations and experiences of patients and spouses. Data were collected through interviews, questionnaires, and clinical measurements from home monitoring devices. Statistical analyses were performed using the IBM SPSS Statistics version 26. Qualitative data were analyzed using NVivo 12.0.

Results

Through interviews, patients articulated the following themes about participating in a TR program: usefulness of the HeartPortal, feeling more secure living with AF, community of practice living with AF, and measuring heart rhythm makes good sense. Through interviews, the spouses of patients with AF expressed that they had gained increased knowledge about AF and how to support their spouses living with AF in everyday life. Results from the responses to the Jessa AF Knowledge Questionnaire support the qualitative data, as they showed that patients in program B acquired increased knowledge about AF at follow-up compared with baseline. No significant differences were found in the number of electrocardiography recordings between the 2 groups.

Conclusions

Patients with AF and their spouses were positive about the TR program and they found the TR program useful, especially because it created an increased sense of security, knowledge about mastering their symptoms, and a community of practice linking patients with AF and their spouses and health care personnel. To assess all the benefits of the Future Patient–TR Program for patients with AF, it needs to be tested in a comprehensive randomized controlled trial.

Trial Registration

ClinicalTrials.gov NCT04493437; https://clinicaltrials.gov/ct2/show/NCT04493437.

Pharmaceutical mobile application for visually-impaired people in Thailand: development and implementation

BACKGROUND

Most mobile pharmaceutical applications produced for people with visual disabilities in Thailand fail to meet the required standard due to poor-quality regulations, defective design, lack of user support and impracticality; as a result, visually-impaired people are unable to use them. This research is motivated by the limited use of this technology in primary medical services and its aim is to enable people with disabilities to access effective digital health information. The research objective is to analyse, design and develop a mobile pharmaceutical application with functions that are appropriate for visually-impaired users, and test its usability.

RESULTS

Based on the design and development of the application, it contained five necessary functions. When testing the usability and users' satisfaction, it was found that the input or fill of information in the application was of low usability. According to the test results, the medicinal database function was missing 71 times and the voice command function was missing 34 times. Based on users' satisfaction results, users who had the highest level of usage gave higher average scores to users' attitude, users' confidence, user interface and system performance than those with lower levels of usage. The scores of both groups were found to be the same when discussing the implementation of the development.

CONCLUSIONS

This mobile application, which was developed based on the use of smart technology, will play an important role in supporting visually-impaired people in Thailand by enhancing the efficacy of self-care. The design and development of the application will ensure the suitability of many functions for visually-impaired users. However, despite the high functional capacity of the application, the gap in healthcare services between the general public and disabled groups will still exist if users have inadequate IT skills.

A Qualitative Study toward Technologies for Active and Healthy Aging: A Thematic Analysis of Perspectives among Primary, Secondary, and Tertiary End Users

It is expected that, by 2050, people aged over 60 in 65 nations will constitute 30% of the total population. Healthy aging is at the top of the world political agenda as a possible means for hindering the collapse of care systems. How can ICT/sensing technology meet older people's needs for active and healthy aging? This qualitative study carried out in Italy and Romania in 2020 involved 30 participants: older adults, caregivers, and stakeholders. Based on a user-centered design approach, this study aimed to understand which requirements of ICT/sensing technologies could match people's needs of active and healthy aging. Findings highlighted that ICT/sensing technology needs to focus on six major themes: (1) learnability, (2) security, (3) independence, empowerment, and coaching values, (4) social isolation, (5) impact of habit, culture, and education variables, and (6) personalized solutions. These themes are consistent with the Active Aging framework and the factors that influence perceived usefulness and potential benefits among older adults. Consequently, this study shows how well-known, but still unresolved, issues affect the field of information and communication technologies (ICTs) to promote active and healthy aging. This suggests that the reinforcement of the public health system, especially considering the pandemic effect, requires a concrete and formidable effort from an interdisciplinary research network.

Patient-centred e-health supports value cocreation and quality of medical care in Taiwan

BACKGROUND

Patient-centred e-health (PCEH) focusses on the interaction between patients and physicians. However, only a limited number of studies have focussed on the design of physician-patient value cocreation mechanisms in the PCEH context. Thus, we extend Grönroos' concept of value cocreation to understand how PCEH might improve the quality of care.

OBJECTIVES

This study proposes a theoretical framework to embody PCEH-supported value cocreation and presents some empirical validation. We expect that PCEH-supported value cocreation should comprise capabilities for patient empowerment, intention for information sharing, complementation for checking and verifying information, and interaction for shared understanding.

METHODS

This study surveyed a small group of patients that have used PCEH, 'My Health Bank' in Taiwan. The questionnaires were delivered to patients in hospitals (n = 167 questionnaires, 98% response rate).

RESULTS

Results indicate that certain PCEH-supported value cocreation mechanisms-capabilities for patient empowerment and interaction for shared understanding-affect the perceived quality of medical care.

LIMITATIONS

The survey only considered patient perceptions of value cocreation.

CONCLUSION

This study shows the patient perception of value cocreation in patient-centred e-Health. Further research needs to validate the framework for health professionals and in other e-Health record information sharing settings.

Virtual Reality Human-Human Interface to Deliver Psychotherapy to People Experiencing Auditory Verbal Hallucinations: Development and Usability Study

BACKGROUND

Digital technologies have expanded the options for delivering psychotherapy, permitting for example, the treatment of schizophrenia using Avatar Therapy. Despite its considerable potential, this treatment method has not been widely disseminated. As a result, its operability and functionality remain largely unknown.

OBJECTIVE

We aimed to study the usability of a therapeutic virtual reality human-human interface, created in a game engine.

METHODS

Participants were psychiatric hospital staff who were introduced to the therapeutic platform in a hands-on session. The System Usability Scale (SUS) was employed for evaluation purposes. Statistical evaluation was conducted using descriptive statistics, the chi-square test, analysis of variance, and multilevel factor analysis.

RESULTS

In total, 109 staff members were introduced to the therapeutic tool and completed the SUS. The mean SUS global score was 81.49 (SD 11.1). Psychotherapists (mean 86.44, SD 8.79) scored significantly higher (F2,106=6.136; P=.003) than nursing staff (mean 79.01, SD 13.30) and administrative personnel (mean 77.98, SD 10.72). A multilevel factor analysis demonstrates a different factor structure for each profession.

CONCLUSIONS

In all professional groups in this study, the usability of a digital psychotherapeutic tool developed using a game engine achieved the benchmark for an excellent system, scoring highest among the professional target group (psychotherapists). The usability of the system seems, to some extent, to be dependent on the professional background of the user. It is possible to create and customize novel psychotherapeutic approaches with gaming technologies and platforms.

TRIAL REGISTRATION

Clinicaltrials.gov NCT04099940; http://clinicaltrials.gov/ct2/show/NCT04099940.

Usefulness of mobile apps for communication of genetic test results to at-risk family members in a U.S. integrated health system: a qualitative approach from user-testing

OBJECTIVE

To assess the usefulness a mobile based application to send genetic test results to at-risk family members in a U.S. integrated health system.

METHODS

We conducted semi-structured in-person interviews with members of Kaiser Permanente Washington who had enrolled in a prospective study and received genetic test results. Participants were given the task to use the app and comment on the experience. The moderator asked participants to share perspectives on the usefulness of a mobile based app and their lived experiences of sharing their test results with family members.

RESULTS

Fourteen study participants who had undergone genetic testing were interviewed. Four primary themes emerged as relevant to the use of mobile-based apps as a tool for communicating genetic test results to at-risk family members: (i) Participants felt a sense of obligation to share positive test results with relatives; (ii) Participants felt that the advantages of using email were similar to those of the app; (iii) Participants felt that younger individuals would be more comfortable with an app; and, (iv) Participants felt they could use the app independently and in their own time.

CONCLUSION

A mobile based app could be used as a tool to improve cascade screening for pathogenic/likely pathogenic test results. The benefits of such a tool are likely greatest among relatives still at the stage of family planning, as well as among family members with strained relationships. There would be minimal burden on the system to offer a mobile based app as a tool.

A framework to lower the risk of medication prescribing and dispensing errors: A usability study of an NFC-based mobile application

BACKGROUND

Wrong medication and wrong dosage are major risks in the pharmaceutical industry, as many medication errors occur when dispensing medication. The dispensing process in its current form is limited in verifying the patient's identity before dispensing the medication. Furthermore, this process does not offer a robust method for providing accurate medication intake instructions. Therefore, we have developed a framework to accurately and securely overcome issues associated with transferring patient credentials and prescription information. The long-term goal of this research is to develop a framework to mitigate medication dispensing errors. One of the framework components is the mobile application that uses near-field communication (NFC) to transfer information. Therefore, in this paper, we designed a user study to assess the proposed NFC-based mobile application in terms of usefulness and ease of use compared with the traditional method of picking up a prescribed medication.

METHODS

We conducted a usability study with 21 participants to perform four tasks to simulate the process of picking up a prescribed medication using the proposed NFC application method and the traditional method of picking up medication. Then, we asked the participants to complete two post-questionnaires after using each method to evaluate the participants' experience of the process. Next, we asked the participants to complete an additional questionnaire about the usefulness of the NFC application method. Finally, we conducted semi-structured interviews with the participants to get more evidence to back up the questionnaire answers.

RESULTS

Our findings show that 91% of the participants believe using the NFC application method will improve patient safety during the medication pickup process. Nearly 97% of participants found the NFC application method easy to use. Our findings show that the participants scored lower when using the NFC application method compared with the traditional method when trying to identify the wrong medication after dispensing. In addition, 90% of the participants successfully identified the wrong medication when using the NFC application method, compared to only 38% when using the traditional method. Finally, the results show that the participants preferred using the NFC application method in terms of information availability, security, and privacy.

CONCLUSIONS

The study findings show that the proposed NFC application for managing patients' prescriptions and picking up medication might improve patient safety. The results show that the participants believe the NFC application will mitigate medication dispensing errors, at least from their end. The participants believe the application will provide a fast and accurate method of verifying dispensed medication from the patient end. Moreover, the application will help the patient to track their current prescription, which also helps them remember the medication intake instructions. Finally, the study indicates that the application will provide a secure, private, and accurate method to help verify the patient's identity, thus minimizing medication errors during the medication dispensing process.

Usability Testing of a Digital Assessment Routing Tool: Protocol for an Iterative Convergent Mixed Methods Study

BACKGROUND

Musculoskeletal conditions account for 16% of global disability, resulting in a negative effect on millions of patients and an increasing burden on health care utilization. Digital technologies that improve health care outcomes and efficiency are considered a priority; however, innovations are often inadequately developed and poorly adopted. Further, they are rarely tested with sufficient rigor in clinical trials-the gold standard for clinical proof of efficacy. We have developed a new musculoskeletal Digital Assessment Routing Tool (DART) that allows users to self-assess and be directed to the right care. DART requires usability testing in preparation for clinical trials.

OBJECTIVE

This study will use the iterative convergent mixed methods design to assess and mitigate all serious usability issues to optimize user experience and adoption. Using this methodology, we will provide justifiable confidence to progress to full-scale randomized controlled trials when DART is integrated into clinical management pathways. This study protocol will provide a blueprint for future usability studies of mobile health solutions.

METHODS

We will collect qualitative and quantitative data from 20-30 participants aged 18 years and older for 4 months. The exact number of participants recruited will be dependent on the number of iterative cycles required to reach the study end points. Building on previous internal testing and stakeholder involvement, quantitative data collection is defined by the constructs within the ISO 9241-210-2019 standard and the system usability scale, providing a usability score for DART. Guided by the participant responses to quantitative questioning, the researcher will focus the qualitative data collection on specific usability problems. These will then be graded to provide the rationale for further DART system improvements throughout the iterative cycles.

RESULTS

This study received approval from the Queen Mary University of London Ethics of Research Committee (QMREC2018/48/048) on June 4, 2020. At manuscript submission, study recruitment was on-going, with data collection to be completed and results published in 2021.

CONCLUSIONS

This study will provide evidence concerning mobile health DART system usability and acceptance determining system improvements required to support user adoption and minimize suboptimal system usability as a potential confounder within subsequent noninferiority clinical trials. Success should produce a safe effective system with excellent usability, facilitating quicker and easier patient access to appropriate care while reducing the burden on primary and secondary care musculoskeletal services. This deliberately rigorous approach to mobile health innovation could be used as a guide for other developers of similar apps.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/27205.

User-Centered Development of a Mobile App for Biopsychosocial Pain Assessment in Adults: Usability, Reliability, and Validity Study

Background

Pain-related mobile apps targeting pain assessment commonly limit pain assessment to pain behaviors and physiological aspects. However, current guidelines state that pain assessment should follow the biopsychosocial model, clearly addressing biological, psychological, and social aspects of the pain experience. Existing reviews also highlight that pain specialists and end users are not commonly involved in the development process of mobile apps for pain assessment, negatively affecting the quality of the available apps.

Objective

This study aimed to develop a mobile app for pain assessment (AvaliaDor) and assess its usability, validity, reliability, and measurement error in a sample of real patients with chronic pain recruited from a physiotherapy clinic.

Methods

This study was divided into 2 phases: phase 1—development of the AvaliaDor app; and phase 2—assessment of the apps’ usability, reliability, measurement error, and validity. AvaliaDor was developed (phase 1) based on the literature and the recommendations of physiotherapists and patients with pain in cycles of evaluation, inclusion of recommendations, and reevaluation until no further changes were required. The final version of the app was then tested in patients with musculoskeletal pain attending a private physiotherapy practice (phase 2) who were asked to use the app twice on 2 consecutive days for reliability purposes. In addition, participants had to complete a set of paper-based scales (Brief Pain Inventory, painDETECT, Pain Catastrophizing Scale, and Tampa Scale for Kinesiophobia), which were used to assess the validity (criterion validity and hypothesis testing) of the app, and the Post-Study System Usability Questionnaire was used to assess its usability.

Results

The development process (phase 1) included 5 physiotherapists external to the research team and 5 patients with musculoskeletal pain, and it resulted in the creation of an app named AvaliaDor, which includes an assessment of pain intensity, location, and phenotype; associated disability; and the issues of pain catastrophizing and fear of movement. A total of 52 patients with pain (mean age 50.12 years, SD 11.71 years; 39 females) participated in phase 2 and used the app twice. The Pearson correlation coefficient between the scores on the paper-based scales and the app ranged between 0.81 and 0.93 for criterion validity and between 0.41 and 0.59 for hypothesis testing. Test-retest reliability was moderate to good (intraclass correlation coefficient between 0.67 and 0.90) and the score for usability was 1.16 (SD 0.27), indicating good usability.

Conclusions

A mobile app named AvaliaDor was developed to assess the intensity, location, and phenotype of pain; associated disability; and the issues of pain catastrophizing and fear of movement in a user-centered design process. The app was shown to be usable, valid, and reliable for assessing pain from a biopsychosocial perspective in a heterogeneous group of patients with pain. Future work can explore the long-term use of AvaliaDor in clinical contexts and its advantages for the assessment and management of patients with pain.

Virtual Reality App for Treating Eating Behavior in Eating Disorders: Development and Usability Study

Background

Anorexia nervosa is one of the more severe eating disorders, which is characterized by reduced food intake, leading to emaciation and psychological maladjustment. Treatment outcomes are often discouraging, with most interventions displaying a recovery rate below 50%, a dropout rate from 20% to 50%, and a high risk of relapse. Patients with anorexia nervosa often display anxiety and aversive behaviors toward food. Virtual reality has been successful in treating vertigo, anxiety disorder, and posttraumatic stress syndrome, and could potentially be used as an aid in treating eating disorders.

Objective

The aim of this study was to evaluate the feasibility and usability of an immersive virtual reality technology administered through an app for use by patients with eating disorders.

Methods

Twenty-six participants, including 19 eating disorder clinic personnel and 5 information technology personnel, were recruited through emails and personal invitations. Participants handled virtual food and utensils on an app using immersive virtual reality technology comprising a headset and two hand controllers. In the app, the participants learned about the available actions through a tutorial and they were introduced to a food challenge. The challenge consisted of a meal type (meatballs, potatoes, sauce, and lingonberries) that is typically difficult for patients with anorexia nervosa to eat in real life. Participants were instructed, via visual feedback from the app, to eat at a healthy rate, which is also a challenge for patients. Participants rated the feasibility and usability of the app by responding to the mHealth Evidence Reporting and Assessment checklist, the 10-item System Usability Scale, and the 20-point heuristic evaluation questionnaire. A cognitive walkthrough was performed using video recordings of participant interactions in the virtual environment.

Results

The mean age of participants was 37.9 (SD 9.7) years. Half of the participants had previous experience with virtual reality. Answers to the mHealth Evidence Reporting and Assessment checklist suggested that implementation of the app would face minor infrastructural, technological, interoperability, financial, and adoption problems. There was some disagreement on intervention delivery, specifically regarding frequency of use; however, most of the participants agreed that the app should be used at least once per week. The app received a mean score of 73.4 (range 55-90), earning an overall “good” rating. The mean score of single items of the heuristic evaluation questionnaire was 3.6 out of 5. The lowest score (2.6) was given to the “accuracy” item. During the cognitive walkthrough, 32% of the participants displayed difficulty in understanding what to do at the initial selection screen. However, after passing the selection screen, all participants understood how to progress through the tasks.

Conclusions

Participants found the app to be usable and eating disorder personnel were positive regarding its fit with current treatment methods. Along with the food item challenges in the current app, participants considered that the app requires improvement to offer environmental and social (eg, crowded room vs eating alone) challenges.

Evaluation of e-health (Seha) application: a cross-sectional study in Saudi Arabia

BACKGROUND

The Ministry of Health in Saudi Arabia is expanding the country's telemedicine services by using advanced technology in health services. In doing so, an e-health application (app), Seha, was introduced in 2018 that allows individuals to have face-to-face visual medical consultations with their doctors on their smartphones.

OBJECTIVE

This study evaluated the effectiveness of the app in improving healthcare delivery by ensuring patient satisfaction with the care given, increasing access to care, and improving efficiency in the healthcare system.

METHODS

A cross-sectional study design was used to assess the perceptions of users of the Seha app and non-users who continued with traditional health services. The data were collected using an online survey via Google Forms between June 2020 and September 2020. Independent t tests and chi-square (χ2) tests were conducted to answer the research questions.

RESULTS

There was a significant difference between users and non-users in terms of ease of access to health services (t =  - 9.38, p < 0.05), with app users having a higher mean score (4.19 ± 0.91) than non-users (3.41 ± 1.00); satisfaction with health services (t =  - 6.33, p < 0.05), with users having a higher mean score (3.96 ± 0.91) than non-users (3.45 ± 0.94); and efficiency (only one visit needed for treatment) (t =  - 3.20, p < 0.05), with users having a higher mean score (3.71 ± 0.93) than non-users (3.45 ± 0.93). There were significant associations between the use of the Seha app and age (χ2 = 8.79, p < 0.05), gender (χ2 = 22.19, p < 0.05), region (χ2 = 19.74, p < 0.05), and occupation (χ2 = 22.05, p < 0.05). There were significant relationships between the three items (on access, satisfaction, and efficiency) and experiencing technical issues (t = 4.47, t = 8.11, and t = 3.24, respectively, p < 0.05), with users who faced technical problems having significantly lower mean scores for all three items.

CONCLUSION

This study provided evidence that the Seha app improved the delivery of healthcare in Saudi Arabia. Users of the app had a better health experience in terms of their perceived ease of access to healthcare services; their satisfaction with healthcare services; and the efficiency of the system, measured by the number of required doctor visits. Other factors that appeared to influence the use of the app included age, gender, usual source of care, and technical difficulties.

Using a Web-Based App to Deliver Rehabilitation Strategies to Persons With Chronic Conditions: Development and Usability Study

Background

The global rise in the incidence of chronic conditions and aging is associated with increased disability. Physiotherapists and occupational therapists can mitigate the resulting burden on the health care system with their expertise in optimizing function. Rehabilitation self-management strategies can assist people with chronic conditions to accept, adjust, and manage different aspects of their daily functioning. Interventions delivered using technology have the potential to increase the accessibility, availability, and affordability of rehabilitation self-management support and services.

Objective

This study aims to describe the development and usability evaluation of iamable, a web-based app created to provide rehabilitation self-management support for people with chronic conditions.

Methods

The development and evaluation of iamable were undertaken in several phases. We used user-centered design principles and an iterative process that included consultations with rehabilitation experts; developed a prototype; and conducted usability tests, heuristic evaluations, and a focus group analysis.

Results

The iamable app was developed to provide rehabilitation self-management strategies in the areas of exercise, fall prevention, fatigue management, pain management, physical activity, and stress management. We engaged adults aged ≥45 years with at least one chronic condition (N=11) in usability testing. They identified navigation and the understanding of instructions as the primary issues for end users. During the heuristic evaluation, clinicians (N=6) recommended that some areas of app content should be more succinct and that help should be more readily available. The focus group provided input to help guide clinical simulation testing, including strategies for selecting patients and overcoming barriers to implementation.

Conclusions

We engaged end users and clinicians in the development and evaluation of the iamable app in an effort to create a web-based tool that was useful to therapists and their patients. By addressing usability issues, we were able to ensure that patients had access to rehabilitation strategies that could be used to help them better manage their health. Our app also provides therapists with a platform that they can trust to empower their patients to be more active in the management of chronic conditions. This paper provides a resource that can be used by others to develop and evaluate web-based health apps.

Early Detection of Dengue Fever Outbreaks Using a Surveillance App (Mozzify): Cross-sectional Mixed Methods Usability Study

Background

While early detection and effective control of epidemics depend on appropriate surveillance methods, the Philippines bases its dengue fever surveillance system on a passive surveillance method (notifications from barangay/village health centers, municipal or city health offices, hospitals, and clinics). There is no available mHealth (mobile health) app for dengue fever that includes all the appropriate surveillance methods in early detection of disease outbreaks in the country.

Objective

This study aimed to evaluate the usability of the Mozzify app in terms of objective quality (engagement, functionality, aesthetics, information) and app subjective and app-specific qualities and compare total app mean score ratings by sociodemographic profile and self and family dengue fever history to see what factors are associated with high app mean score rating among school-based young adult samples and health care professionals. Individual interviews and focus group discussions were also conducted among participants to develop themes from their comments and suggestions to help structure further improvement and future development of the app.

Methods

User experience sessions were conducted among participants, and the Mobile Application Rating Scale (MARS) professional and user versions (uMARS) were administered followed by individual interviews and focus group discussions. Descriptive statistical analysis of the MARS and uMARS score ratings was performed. The total app mean score ratings by sociodemographic and dengue fever history using nonparametric mean difference analyses were also conducted. Thematic synthesis was used to develop themes from the comments and suggestions raised in individual interviews and focus group discussions.

Results

Mozzify obtained an overall >4 (out of 5) mean score ratings in the MARS and uMARS app objective quality (4.45), subjective (4.17), and specific (4.55) scales among 948 participants (79 health care professionals and 869 school-based samples). Mean difference analyses revealed that total app mean score ratings were not significantly different across ages and gender among health care professionals and across age, income categories, and self and family dengue fever history but not gender (P<.001) among the school-based samples. Thematic syntheses revealed 7 major themes: multilanguage options and including other diseases; Android version availability; improvements on the app’s content, design, and engagement; inclusion of users from low-income and rural areas; Wi-Fi connection and app size concerns; data credibility and issues regarding user security and privacy.

Conclusions

With its acceptable performance as perceived by health care professionals and school-based young adults, Mozzify has the potential to be used as a strategic health intervention system for early detection of disease outbreaks in the Philippines. It can be used by health care professionals of any age and gender and by school-based samples of any age, socioeconomic status, and dengue fever history. The study also highlights the feasibility of school-based young adults to use health-related apps for disease prevention.

Opinions of African American adults about the use of apolipoprotein L1 (ApoL1) genetic testing in living kidney donation and transplantation

Apolipoprotein L1 (ApoL1) predictive genetic testing for kidney disease, and its emerging role in transplantation, remains controversial as it may exacerbate underlying disparities among African Americans (AAs) at increased risk. We conducted an online simulation among AAs (N = 585) about interest in ApoL1 testing and its cofactors, under 2 scenarios: as a potential living donor (PLD), and as a patient awaiting transplantation. Most respondents (61%) expressed high interest in genetic testing as a PLD: age ≥35 years (adjusted odds ratio [aOR], 1.75; 95% confidence interval [CI], 1.18, 2.60, P = .01), AA identity (aOR, 1.67; 95% CI, 1.02, 2.72, P = .04), perceived kidney disease risk following donation (aOR, 1.68; 95% CI, 1.03, 2.73, P = .03), interest in genetics (aOR, 2.89; 95% CI, 1.95, 4.29, P = .001), and genetics self-efficacy (aOR, 2.38; 95% CI, 1.54, 3.67, P = .001) were positively associated with ApoL1 test interest. If awaiting transplantation, most (89%) believed that ApoL1 testing should be done on AA deceased donors, and older age (aOR, 1.85; 95% CI, 1.03, 3.32, P = .04) and greater interest in genetics (aOR, 2.61; 95% CI, 1.41, 4.81, P = .002) were associated with interest in testing deceased donors. Findings highlight strong support for ApoL1 testing in AAs and the need to examine such opinions among PLDs and transplant patients to enhance patient education efforts.

Acceptability, Validity, and Engagement With a Mobile App for Frequent, Continuous Multiyear Assessment of Youth Health Behaviors (mNCANDA): Mixed Methods Study

BACKGROUND

Longitudinal studies of many health behaviors often rely on infrequent self-report assessments. The measurement of psychoactive substance use among youth is expected to improve with more frequent mobile assessments, which can reduce recall bias. Researchers have used mobile devices for longitudinal research, but studies that last years and assess youth continuously at a fine-grained, temporal level (eg, weekly) are rare. A tailored mobile app (mNCANDA [mobile National Consortium on Alcohol and Neurodevelopment in Adolescence]) and a brief assessment protocol were designed specifically to provide a feasible platform to elicit responses to health behavior assessments in longitudinal studies, including NCANDA (National Consortium on Alcohol and Neurodevelopment in Adolescence).

OBJECTIVE

This study aimed to determine whether an acceptable mobile app system could provide repeatable and valid assessment of youth's health behaviors in different developmental stages over extended follow-up.

METHODS

Participants were recruited (n=534; aged 17-28 years) from a larger longitudinal study of neurodevelopment. Participants used mNCANDA to register reports of their behaviors for up to 18 months. Response rates as a function of time measured using mNCANDA and participant age were modeled using generalized estimating equations to evaluate response rate stability and age effects. Substance use reports captured using mNCANDA were compared with responses from standardized interviews to assess concurrent validity. Reactivity was assessed by evaluating patterns of change in substance use after participants initiated weekly reports via mNCANDA. Quantitative feedback about the app was obtained from the participants. Qualitative interviews were conducted with a subset of participants who used the app for at least one month to obtain feedback on user experience, user-derived explanations of some quantitative results, and suggestions for system improvements.

RESULTS

The mNCANDA protocol adherence was high (mean response rate 82%, SD 27%) and stable over time across all age groups. The median time to complete each assessment was 51 s (mean response time 1.14, SD 1.03 min). Comparisons between mNCANDA and interview self-reports on recent (previous 30 days) alcohol and cannabis use days demonstrate close agreement (eg, within 1 day of reported use) for most observations. Models used to identify reactivity failed to detect changes in substance use patterns subsequent to enrolling in mNCANDA app assessments (P>.39). Most participants (64/76, 84%) across the age range reported finding the mNCANDA system acceptable. Participants provided recommendations for improving the system (eg, tailoring signaling times).

CONCLUSIONS

mNCANDA provides a feasible, multi-year, continuous, fine-grained (eg, weekly) assessment of health behaviors designed to minimize respondent burden and provides acceptable regimes for long-term self-reporting of health behaviors. Fine-grained characterization of variability in behaviors over relatively long periods (eg, up to 18 months) may, through the use of mNCANDA, improve our understanding of the relationship between substance use exposure and neurocognitive development.

Using a Commercially Available App for the Self-Management of Hypertension: Acceptance and Usability Study in Saudi Arabia

BACKGROUND

The use of smartphone apps to assist in the self-management of hypertension is becoming increasingly common, but few commercially available apps have the potential to be effective along with adequate security and privacy measures in place. In a previous study, we identified 5 apps that are potentially effective and safe, and based on the preferences of doctors and patients, one (Cora Health) was selected as the most suitable app for use in a Saudi context. However, there is currently no evidence of its usability and acceptance among potential users. Indeed, there has been little research into the usability and acceptance of hypertension apps in general, and less research considers this in the Gulf Region.

OBJECTIVE

This study aims to evaluate the acceptance and usability of the selected app in the Saudi context.

METHODS

This study used a mixed methods approach with 2 studies: a usability test involving patients in a controlled setting performing predefined tasks and a real-world usability study where patients used the app for 4 weeks. In the usability test, participants were asked to think aloud while performing the tasks, and an observer recorded the number of tasks they completed. At the end of the real-world pilot study, participants were interviewed, and the mHealth App Usability Questionnaire was completed. Descriptive statistics were used to analyze quantitative data, and thematic analysis was used to analyze qualitative data.

RESULTS

In total, 10 patients completed study 1. The study found that app usability was moderate and that participants needed some familiarization time before they could use the app proficiently. Some usability issues were revealed, related to app accessibility and navigation, and a few tasks remained uncompleted by most people. A total of 20 patients completed study 2, with a mean age of 51.6 (SD 11.7) years. Study 2 found that the app was generally acceptable and easy to use, with some similar usability issues identified. Participants stressed the importance of practice and training to use it more easily and proficiently. Participants had a good engagement level with 48% retention at the end of study 2, with most participants' engagement being classed as meaningful. The most recorded data were blood pressure, followed by stress and medication, and the most accessed feature was viewing graphs of data trends.

CONCLUSIONS

This study shows that a commercially available app can be usable and acceptable in the self-management of hypertension but also found a considerable number of possibilities for improvement, which needs to be considered in future app development. The results show that there is potential for a commercially available app to be used in large-scale studies of hypertension self-management if suggestions for improvements are addressed.

Exploring the effectiveness of telehealth interventions for diagnosis, contact tracing and care of Corona Virus Disease of 2019 (COVID19) patients in sub Saharan Africa: a rapid review

The efficacy of leveraging telehealth services on clinical outcomes remains scarcely documented. We conducted a rapid review to explore the effectiveness of telehealth interventions for the diagnosis, contact tracing and care of the corona virus disease of 2019 in sub Saharan Africa. Using MEDLINE, Science Direct and Cochrane Library databases, a review was conducted during the month of July 2020 of peer reviewed articles reporting on the use of telehealth interventions in sub-Saharan Africa. All the studies were assessed against the inclusion criteria by two independent reviewers. The 7 studies included in the synthesis were conducted in 2 countries [Nigeria (× 1), Zimbabwe (× 1)], and commentaries covering the entire sub Saharan Africa in general (× 4) and to Uganda (× 1). All the included articles and commentaries were published in 2020. We established that mobile applications are effective in providing information for referrals of potential patients infected by COVID 19 and provides convenient access to routine care without the risk of exposure through close contact. In countries such as Nigeria, mobile positioning data significantly improved decision making, capacity and scope of contact tracing and surveillance of known contacts of confirmed cases. We noted that collaborations between the government, mobile network operators and technology companies were utilised for successful mobile positioning data tracing interventions for COVID patients. Mobile applications such as the Vula platform in South Africa were also noted as effective in providing psychological support to health care workers attending to patients.

Malay Version of the mHealth App Usability Questionnaire (M-MAUQ): Translation, Adaptation, and Validation Study

BACKGROUND

Mobile health (mHealth) apps play an important role in delivering education, providing advice on treatment, and monitoring patients' health. Good usability of mHealth apps is essential to achieve the objectives of mHealth apps efficiently. To date, there are questionnaires available to assess the general system usability but not explicitly tailored to precisely assess the usability of mHealth apps. Hence, the mHealth App Usability Questionnaire (MAUQ) was developed with 4 versions according to the type of app (interactive or standalone) and according to the target user (patient or provider). Standalone MAUQ for patients comprises 3 subscales, which are ease of use, interface and satisfaction, and usefulness.

OBJECTIVE

This study aimed to translate and validate the English version of MAUQ (standalone for patients) into a Malay version of MAUQ (M-MAUQ) for mHealth app research and usage in future in Malaysia.

METHODS

Forward and backward translation and harmonization of M-MAUQ were conducted by Malay native speakers who also spoke English as their second language. The process began with a forward translation by 2 independent translators followed by harmonization to produce an initial translated version of M-MAUQ. Next, the forward translation was continued by another 2 translators who had never seen the original MAUQ. Lastly, harmonization was conducted among the committee members to resolve any ambiguity and inconsistency in the words and sentences of the items derived with the prefinal adapted questionnaire. Subsequently, content and face validations were performed with 10 experts and 10 target users, respectively. Modified kappa statistic was used to determine the interrater agreement among the raters. The reliability of the M-MAUQ was assessed by 51 healthy young adult mobile phone users. Participants needed to install the MyFitnessPal app and use it for 2 days for familiarization before completing the designated task and answer the M-MAUQ. The MyFitnessPal app was selected because it is one among the most popular installed mHealth apps globally available for iPhone and Android users and represents a standalone mHealth app.

RESULTS

The content validity index for the relevancy and clarity of M-MAUQ were determined to be 0.983 and 0.944, respectively, which indicated good relevancy and clarity. The face validity index for understandability was 0.961, which indicated that users understood the M-MAUQ. The kappa statistic for every item in M-MAUQ indicated excellent agreement between the raters (κ ranging from 0.76 to 1.09). The Cronbach α for 18 items was .946, which also indicated good reliability in assessing the usability of the mHealth app.

CONCLUSIONS

The M-MAUQ fulfilled the validation criteria as it revealed good reliability and validity similar to the original version. M-MAUQ can be used to assess the usability of mHealth apps in Malay in the future.

Usability and Preliminary Effectiveness of a Preoperative mHealth App for People Undergoing Major Surgery: Pilot Randomized Controlled Trial

BACKGROUND

Major surgery is associated with negative postoperative outcomes such as complications and delayed or poor recovery. Multimodal prehabilitation can help to reduce the negative effects of major surgery. Offering prehabilitation by means of mobile health (mHealth) could be an effective new approach.

OBJECTIVE

The objectives of this pilot study were to (1) evaluate the usability of the Be Prepared mHealth app prototype for people undergoing major surgery, (2) explore whether the app was capable of bringing about a change in risk behaviors, and (3) estimate a preliminary effect of the app on functional recovery after major surgery.

METHODS

A mixed-methods pilot randomized controlled trial was conducted in two Dutch academic hospitals. In total, 86 people undergoing major surgery participated. Participants in the intervention group received access to the Be Prepared app, a smartphone app using behavior change techniques to address risk behavior prior to surgery. Both groups received care as usual. Usability (System Usability Scale), change in risk behaviors 3 days prior to surgery, and functional recovery 30 days after discharge from hospital (Patient-Reported Outcomes Measurement Information System physical functioning 8-item short form) were assessed using online questionnaires. Quantitative data were analyzed using descriptive statistics, chi-square tests, and multivariable linear regression. Semistructured interviews about the usability of the app were conducted with 12 participants in the intervention group. Thematic analysis was used to analyze qualitative data.

RESULTS

Seventy-nine people-40 in the intervention group and 39 in the control group-were available for further analysis. Participants had a median age of 61 (interquartile range 51.0-68.0) years. The System Usability Scale showed that patients considered the Be Prepared app to have acceptable usability (mean 68.2 [SD 18.4]). Interviews supported the usability of the app. The major point of improvement identified was further personalization of the app. Compared with the control group, the intervention group showed an increase in self-reported physical activity and muscle strengthening activities prior to surgery. Also, 2 of 2 frequent alcohol users in the intervention group versus 1 of 9 in the control group drank less alcohol in the run-up to surgery. No difference was found in change of smoking cessation. Between-group analysis showed no meaningful differences in functional recovery after correction for baseline values (β=-2.4 [95% CI -5.9 to 1.1]).

CONCLUSIONS

The Be Prepared app prototype shows potential in terms of usability and changing risk behavior prior to major surgery. No preliminary effect of the app on functional recovery was found. Points of improvement have been identified with which the app and future research can be optimized.

TRIAL REGISTRATION

Netherlands Trial Registry NL8623; https://www.trialregister.nl/trial/8623.

Spanish Version of the System Usability Scale for the Assessment of Electronic Tools: Development and Validation

Background

The System Usability Scale (SUS) is a common metric used to assess the usability of a system, and it was initially developed in English. The implementation of electronic systems for clinical counseling (eHealth and mobile health) is increasing worldwide. Therefore, tools are needed to evaluate these applications in the languages and regional contexts in which the electronic tools are developed.

Objective

This study aims to translate, culturally adapt, and validate the original English version of the SUS into a Spanish version.

Methods

The translation process included forward and backward translation. Forward translations were made by 2 native Spanish speakers who spoke English as their second language, and a backward translation was made by a native English speaker. The Spanish SUS questionnaire was validated by 10 experts in mobile app development. The face validity of the questionnaire was tested with 10 mobile phone users, and the reliability testing was conducted among 88 electronic application users.

Results

The content validity index of the new Spanish SUS was good, as indicated by a rating of 0.92 for the relevance of the items. The questionnaire was easy to understand, based on a face validity index of 0.94. The Cronbach α was .812 (95% CI 0.748-0.866; P<.001).

Conclusions

The new Spanish SUS questionnaire is a valid and reliable tool to assess the usability of electronic tools among Spanish-speaking users.

The Role of Health Kiosks: A Scoping Review (Preprint)

Background

Health kiosks are publicly accessible computing devices that provide access to services, including health information provision, clinical measurement collection, patient self–check-in, telemonitoring, and teleconsultation. Although the increase in internet access and ownership of smart personal devices could make kiosks redundant, recent reports have predicted that the market will continue to grow.

Objective

We seek to clarify the current and future roles of health kiosks by investigating the settings, roles, and clinical domains in which kiosks are used; whether usability evaluations of health kiosks are being reported, and if so, what methods are being used; and what the barriers and facilitators are for the deployment of kiosks.

Methods

We conducted a scoping review using a bibliographic search of Google Scholar, PubMed, and Web of Science databases for studies and other publications between January 2009 and June 2020. Eligible papers described the implementation as primary studies, systematic reviews, or news and feature articles. Additional reports were obtained by manual searching and querying the key informants. For each article, we abstracted settings, purposes, health domains, whether the kiosk was opportunistic or integrated with a clinical pathway, and whether the kiosk included usability testing. We then summarized the data in frequency tables.

Results

A total of 141 articles were included, of which 134 (95%) were primary studies, and 7 (5%) were reviews. Approximately 47% (63/134) of the primary studies described kiosks in secondary care settings. Other settings included community (32/134, 23.9%), primary care (24/134, 17.9%), and pharmacies (8/134, 6%). The most common roles of the health kiosks were providing health information (47/134, 35.1%), taking clinical measurements (28/134, 20.9%), screening (17/134, 12.7%), telehealth (11/134, 8.2%), and patient registration (8/134, 6.0%). The 5 most frequent health domains were multiple conditions (33/134, 24.6%), HIV (10/134, 7.5%), hypertension (10/134, 7.5%), pediatric injuries (7/134, 5.2%), health and well-being (6/134, 4.5%), and drug monitoring (6/134, 4.5%). Kiosks were integrated into the clinical pathway in 70.1% (94/134) of studies, opportunistic kiosks accounted for 23.9% (32/134) of studies, and in 6% (8/134) of studies, kiosks were used in both. Usability evaluations of kiosks were reported in 20.1% (27/134) of papers. Barriers (e.g., use of expensive proprietary software) and enablers (e.g., handling of on-demand consultations) of deploying health kiosks were identified.

Conclusions

Health kiosks still play a vital role in the health care system, including collecting clinical measurements and providing access to web-based health services and information to those with little or no digital literacy skills and others without personal internet access. We identified research gaps, such as training needs for teleconsultations and scant reporting on usability evaluation methods.

Engagement Features in Physical Activity Smartphone Apps: Focus Group Study With Sedentary People

Background

Engagement with physical activity mobile apps has been reported to be a core precondition for their effectiveness in digital behavior change interventions. However, to date, little attention has been paid to understanding the perspectives, needs, expectations, and experiences of potential users with physical activity mobile apps.

Objective

The aim of this study was to investigate the features that are judged to be important for engagement with a physical activity mobile app and the reasons for their importance.

Methods

A qualitative focus-group methodology with elements of co-design was adopted in this study. Participants reporting sedentary lifestyles and willingness to improve their physical activity behavior through mobile technology were recruited. The focus group sessions consisted of 13 participants (8 men and 5 women, mean [SD] age 41.9 [7.1] years). Two researchers conducted the data analysis independently by using the inductive thematic approach.

Results

Four main themes emerged in relation to the research question and were named as follows: “physical activity participation motives,” “autonomy and self-regulation,” “need for relatedness,” and “smart.” Additionally, 2 subthemes originated from “physical activity participation motives” (ie, “medical guidance” and “weight loss and fitness for health”) and “smart” (ie, “action planning” and “adaptable and tailored”).

Conclusions

Features enhancing autonomy and self-regulation and positively affecting health and physical well-being as well as the need for relatedness, adaptability, and flexibility should be considered as core elements in the engagement of potential users with physical activity mobile apps. The emerged findings may orient future research and interventions aiming to foster engagement of potential users with physical activity apps.

Developing and evaluating ASTHMAXcel adventures: A novel gamified mobile application for pediatric patients with asthma

BACKGROUND

The ASTHMAXcel mobile application has been linked to favorable outcomes among adult patients with asthma.

OBJECTIVE

To assess the impact of ASTHMAXcel Adventures, a gamified, guideline-based, pediatric version on asthma control, knowledge, health care utilization, and patient satisfaction.

METHODS

Pediatric patients with asthma received the ASTHMAXcel Adventures mobile intervention on-site only at baseline (visit 1), 4 months (visit 2), and 6 months (visit 3). The asthma control test, asthma illness representation scale-self-administered, pediatric asthma impact survey, and Client Satisfaction Questionnaire-8 were used to assess asthma control, knowledge, and patient satisfaction. Patients reported the number of asthma-related emergency department (ED) visits, hospitalizations, and oral prednisone use.

RESULTS

A total of 39 patients completed the study. The proportion of controlled asthma increased from visit 1 to visits 2 and 3 (30.8% vs 53.9%, P = .04; 30.8% vs 59.0%, P = .02), and largely seen in boys. The mean asthma illness representation scale-self-administered scores increased from baseline pre- to postintervention, with sustained improvements at visits 2 and 3 (3.55 vs 3.76, P < .001; 3.55 vs 3.80, P = .001; 3.55 vs 3.99, P < .001). The pediatric asthma impact survey scores improved from baseline to visits 2 and 3 (43.33 vs 34.08, P < .001; 43.33 vs 31.74, P < .001). ED visits and prednisone use significantly decreased from baseline to visits 2 and 3 (ED: 0.46 vs 0.13, P = .03; 0.46 vs 0.02, P = .02; prednisone use, 0.49 vs 0.13, P = .02; 0.49 vs 0.03, P = .003. Satisfaction was high with mean client satisfaction questionnaire score of approximately 30 (out of 32) at all visits.

CONCLUSION

ASTHMAXcel Adventures improved asthma control, knowledge, and quality of life, and reduced ED visits and prednisone use with high satisfaction scores.

Mobile Health and Inhaler-Based Monitoring Devices for Asthma Management

Mobile health and web applications (apps), wearables, and other personal monitoring devices have tremendous potential to improve the management of asthma. More than 500 asthma-related apps, whether standalone or paired with sensors on inhalers, are currently available for health education, symptom recording, tracking of inhaler use, displaying environmental alerts, and providing medication reminders. Benefits of these tools include the ability to longitudinally collect symptom, trigger, and inhaler usage data, allowing the detection of significant changes over time to help patients and their caregivers determine whether symptoms are worsening. In addition, data from external information sources, including weather, allergen, and air quality reports, can be integrated with user-specific data to enhance predictions on when patients may experience symptoms and/or need to avoid triggers. Barriers to adoption of asthma-related apps and inhaler-based devices include uncertain efficacy and effectiveness, potential high cost, sustained user engagement, and concerns about privacy. Moreover, ensuring the acceptability and utility of asthma management apps for individuals of all races/ethnicities, socioeconomic groups, ages, genders, and literacy levels is necessary. Based on studies thus far, mobile health apps and inhaler-based devices have great potential to serve as useful tools in the patient-doctor relationship and revolutionize asthma care.

Feasibility-Usability Study of a Tablet App Adapted Specifically for Persons with Cognitive Impairment-SMART4MD (Support Monitoring and Reminder Technology for Mild Dementia)

Population ageing within Europe has major social and economic consequences. One of the most devastating conditions that predominantly affects older people is dementia. The SMART4MD (Support Monitoring and Reminder Technology for Mild Dementia) project aims to develop and test a health application specifically designed for people with mild dementia. The aim of this feasibility study was to evaluate the design of the SMART4MD protocol, including recruitment, screening, baseline examination and data management, and to test the SMART4MD application for functionality and usability before utilization in a full-scale study. The feasibility study tested the protocol and the app in Spain and Sweden. A total of nineteen persons with cognitive impairment, and their informal caregivers, individually performed a task-based usability test of the SMART4MD app model in a clinical environment, followed by four-week testing of the app in the home environment. By employing a user-centered design approach, the SMART4MD application proved to be an adequate and feasible interface for an eHealth intervention. In the final usability test, a score of 81% satisfied users was obtained. The possibility to test the application in all the procedures included in the study generated important information on how to present the technology to the users and how to improve these procedures.

A Digital Coach Promoting Healthy Aging among Older Adults in Transition to Retirement: Results from a Qualitative Study in Italy

Global aging and increasing multimorbidity are questioning the sustainability of healthcare systems. Healthy aging is at the top of the world political agenda, as a possible means for hindering the collapse of care systems. In the aging process, the transition to retirement can lead to an improvement or a deterioration of physical and psychological health. Digital health coaching technology can support older adults at this stage, but what must be the role of such a solution in promoting healthy aging and shaping sustainable care? This qualitative study, carried out in Italy in 2019, involved 15 older workers, retirees, and colleagues. Based on a user-centered design approach, this study aims at gathering older adults’ feelings on a digital health coaching technology for exploring this solution’s potential in promoting healthy aging. Findings highlighted that the digital health coach may help older adults improve several health determinants, e.g., physical activity, cognitive capabilities, and social life, but it can also entail the risk of stigma and break people’s privacy. The latter can be guaranteed by technology customization and codesign. Further research on the digital health coach benefits to boost healthy aging is needed to understand its potential for shaping future sustainable healthcare.

Usability, acceptability, and feasibility of the Implementation of Infant Pain Practice Change (ImPaC) Resource

The Implementation of Infant Pain Practice (ImPaC) Resource is an eHealth tool designed to support infant pain practice change and ultimately enhance pain outcomes. The aim of this study was to determine users' perspectives on usability, acceptability, and feasibility of the ImPaC Resource. A descriptive prospective mixed‐methods quality improvement study was conducted at a pediatric hospital in Canada. Individual “think aloud” interviews were conducted in a nonclinical environment (Phase A); “near live” testing was conducted while users interacted with the Resource in clinical setting (Phase B); individual “think‐aloud” interviews were conducted in a nonclinical environment (Phase C). Outcomes included usability (System Usability Scale—SUS), acceptability (Acceptability E‐Scale—AES), and feasibility. Interview transcripts were coded per a priori themes using deductive content analysis to create a structured categorization matrix. In Phase A, 10 clinicians interacted with the Resource in individual sessions. Median SUS score was 73.75 (range 52.5‐92.5). In Phase B, four clinicians implemented the Resource in the neonatal intensive care unit (NICU) over 4 months. Median SUS score was 85 (82.5‐92.5), and median AES score was 24 (21‐24). In Phase C, an enhanced prototype was produced, and the same users from Phase B navigated the Resource in individual sessions. Median SUS score was 88.75 (85‐95), and median AES score was 27.5 (25‐29). Users considered the Resource as feasible for implementation, easy to navigate, engaging, intuitive, comprehensive, and evidence‐based. Users highlighted the potential transferability of the Resource to other contexts and settings. The enhanced version of the ImPaC Resource was usable, acceptable, feasible, and met users' expectations and requirements. Results lead the way for evaluation of the Resource in a nationwide cluster randomized trial including 18 NICUs. This knowledge‐rich platform is expected to enhance infant pain practices and outcomes in diverse clinical settings.

Challenges for the evaluation of digital health solutions-A call for innovative evidence generation approaches

The field of digital health, and its meaning, has evolved rapidly over the last 20 years. For this article we followed the most recent definition provided by FDA in 2020. Emerging solutions offers tremendous potential to positively transform the healthcare sector. Despite the growing number of applications, however, the evolution of methodologies to perform timely, cost-effective and robust evaluations have not kept pace. It remains an industry-wide challenge to provide credible evidence, therefore, hindering wider adoption. Conventional methodologies, such as clinical trials, have seldom been applied and more pragmatic approaches are needed. In response, several academic centers such as researchers from the Institute of Global Health Innovation at Imperial College London have initiated a digital health clinical simulation test bed to explore new approaches for evidence gathering relevant to solution type and maturity. The aim of this article is to: (1) Review current research approaches and discuss their limitations; (2) Discuss challenges faced by different stakeholders in undertaking evaluations; and (3) Call for new approaches to facilitate the safe and responsible growth of the digital health sector.

Usability Testing a Potentially Inappropriate Medication Dashboard: A Core Component of the Dashboard Development Process

Background  With the increased usage of dashboard reporting systems to monitor and track patient panels by clinical users, developers must ensure that the information displays they produce are accurate and intuitive. When evaluating usability of a clinical dashboard among potential end users, developers oftentimes rely on methods such as questionnaires as opposed to other, more time-intensive strategies that incorporate direct observation.

Objectives  Prior to release of the potentially inappropriate medication (PIM) clinical dashboard, designed to facilitate completion of a quality improvement project by clinician scholars enrolled in the Veterans Affairs (VA) workforce development Geriatric Scholars Program (GSP), we evaluated the usability of the system. This article describes the process of usability testing a dashboard reporting system with clinicians using direct observation and think-aloud moderating techniques.

Methods  We developed a structured interview protocol that combines virtual observation, think-aloud moderating techniques, and retrospective questioning of the overall user experience, including use of the System Usability Scale (SUS). Thematic analysis was used to analyze field notes from the interviews of three GSP alumni.

Results  Our structured approach to usability testing identified specific functional problems with the dashboard reporting system that were missed by results from the SUS. Usability testing lead to overall improvements in the intuitive use of the system, increased data transparency, and clarification of the dashboard's purpose.

Conclusion  Reliance solely on questionnaires and surveys at the end stages of dashboard development can mask potential functional problems that will impede proper usage and lead to misinterpretation of results. A structured approach to usability testing in the developmental phase is an important tool for developers of clinician friendly systems for displaying easily digested information and tracking outcomes for the purpose of quality improvement.