The Iterative Convergent Design for Mobile Health Usability Testing: Mixed Methods Approach

Evaluating the Usability of the Wellness School Assessment Tool Whole School, Whole Community, Whole Child (WellSAT WSCC): A School Wellness Policy Assessment Tool

BACKGROUND

Adoption of the Whole School, Whole Community, Whole Child (WSCC) model has been slowed by a lack of available tools to support implementation. The Wellness School Assessment Tool (WellSAT) WSCC is an online assessment tool that allows schools to evaluate the alignment of their policies with the WSCC model. This study assesses the usability of the WellSAT WSCC.

METHODS

Using a convergent mixed methods design, we collected qualitative and quantitative data from 5 school-based participants with roles in development and evaluation of policy. Participants explored the platform while engaging in a think-aloud procedure and scored a sample policy using the platform. They also completed the System Usability Scale and responded to open-ended questions about the usability of the platform.

RESULTS

Participants rated the WellSAT WSCC as an above-average user experience, but data suggested several areas for improvement, including improved instructions, enhanced visual design of the platform, and guidance for subsequent policy changes.

CONCLUSION

The WellSAT WSCC provides an above-average user experience but can be improved to increase user experience. These improvements increase the potential for greater use to facilitate integration of the WSCC model into school policy.

A Self-Guided Web-Based App (MyDiaMate) for Enhancing Mental Health in Adults With Type 1 Diabetes: Insights From a Real-World Study in the Netherlands

BACKGROUND

MyDiaMate is a web-based intervention specifically designed for adults with type 1 diabetes (T1D) that aims to help them improve and maintain their mental health. Prior pilot-testing of MyDiaMate verified its acceptability, feasibility, and usability.

OBJECTIVE

This study aimed to investigate the real-world uptake and usage of MyDiaMate in the Netherlands.

METHODS

Between March 2021 and December 2022, MyDiaMate was made freely available to Dutch adults with T1D. Usage (participation and completion rates of the modules) was tracked using log data. Users could volunteer to participate in the user profile study, which required filling out a set of baseline questionnaires. The usage of study participants was examined separately for participants scoring above and below the cutoffs of the "Problem Areas in Diabetes" (PAID-11) questionnaire (diabetes distress), the "World Health Organization Well-being Index" (WHO-5) questionnaire (emotional well-being), and the fatigue severity subscale of the "Checklist Individual Strength" (CIS) questionnaire (fatigue). Two months after creating an account, study participants received an evaluation questionnaire to provide us with feedback.

RESULTS

In total, 1008 adults created a MyDiaMate account, of whom 343 (34%) participated in the user profile study. The mean age was 43 (SD 14.9; 18-76) years. Most participants were female (n=217, 63.3%) and higher educated (n=198, 57.6%). The majority had been living with T1D for over 5 years (n=241, 73.5%). Of the study participants, 59.1% (n=199) of them reported low emotional well-being (WHO-5 score≤50), 70.9% (n=239) of them reported elevated diabetes distress (PAID-11 score≥18), and 52.4% (n=178) of them reported severe fatigue (CIS score≥35). Participation rates varied between 9.5% (n=19) for social environment to 100% (n=726) for diabetes in balance, which opened by default. Completion rates ranged from 4.3% (n=1) for energy, an extensive cognitive behavioral therapy module, to 68.6% (n=24) for the shorter module on hypos. There were no differences in terms of participation and completion rates of the modules between study participants with a more severe profile, that is, lower emotional well-being, greater diabetes distress, or more fatigue symptoms, and those with a less severe profile. Further, no technical problems were reported, and various suggestions were made by study participants to improve the application, suggesting a need for more personalization.

CONCLUSIONS

Data from this naturalistic study demonstrated the potential of MyDiaMate as a self-help tool for adults with T1D, supplementary to ongoing diabetes care, to improve healthy coping with diabetes and mental health. Future research is needed to explore engagement strategies and test the efficacy of MyDiaMate in a randomized controlled trial.

Usability testing of a mobile health application to support individuals with active tuberculosis: a mixed methods study

Poor adherence to tuberculosis (TB) treatment leads to further disease transmission, worsened outcomes, and the development of drug resistance. Digital adherence technologies may facilitate a more patient-centered approach for improving TB treatment outcomes than current strategies. The objective of this study was to evaluate and explore improving usability of the TB Treatment Support Tools (TB-TST) mobile application. We used an iterative convergent mixed-method design consisting of two quantitative surveys and a qualitative think-aloud interview. Testing was conducted in three testing cycles consisting of a total of 16 interviews and 26 surveys. Results were thematically analyzed and reported to the development team during weekly team meetings. Participants rated the TB-TSTs application as having high usability and the iterative approach resulted in several refinements to the application in response to participant feedback. These refinements were well received during qualitative interviews but did not result in a statistically significant improvement in usability testing scores between cycles. Using an iterative convergent mixed-method design was an effective method for refining our mHealth application. Data collected from think-aloud interviews, the MAUQ, and the Health-ITUES identified key areas of application design that needed refinement.

Usability and Evaluation of a Health Information System in the Emergency Department: Mixed Methods Study

BACKGROUND

A lack of information during an emergency visit leads to the experience of powerlessness for patients and their family members, who may also feel unprepared to cope with acute symptoms. The ever-changing nature and fast-paced workflow in the emergency department (ED) often affect how health care professionals can tailor information and communication to the needs of the patient.

OBJECTIVE

This study aimed to evaluate the usability and experience of a newly developed information system. The system was developed together with patients and their family members to help provide the information needed in the ED.

METHODS

We conducted a mixed methods study consisting of quantitative data obtained from the System Usability Scale questionnaire and qualitative interview data obtained from purposively selected participants included in the quantitative part of the study.

RESULTS

A total of 106 patients and 14 family members (N=120) answered the questionnaire. A total of 10 patients and 3 family members participated in the interviews. Based on the System Usability Scale score, the information system was rated close to excellent, with a mean score of 83.6 (SD 12.8). Most of the participants found the information system easy to use and would like to use it again. The participants reported that the system helped them feel in control, and the information was useful. Simplifications were needed to improve the user experience for the older individuals.

CONCLUSIONS

This study demonstrates that the usability of the information system is rated close to excellent. It was perceived to be useful as it enabled understanding and predictability of the patient's trajectory in the ED. Areas for improvement include making the system more usable by older individuals. The study provides an example of how a technological solution can be used to diminish the information gap in an ED context.

From digital health to learning health systems: four approaches to using data for digital health design

Digital health technologies, powered by digital data, provide an opportunity to improve the efficacy and efficiency of health systems at large. However, little is known about different approaches to the use of data for digital health design, or about their possible relations to system-level dynamics. In this contribution, we identify four existing approaches to the use of data for digital health design, namely the silent, the overt, the data-enabled, and the convergent. After characterising the approaches, we provide real-life examples of each. Furthermore, we compare the approaches in terms of selected desirable characteristics of the design process, highlighting relative advantages and disadvantages. Finally, we reflect on the system-level relevance of the differentiation between the approaches and point towards future research directions. Overall, the contribution provides researchers and practitioners with a broad conceptual framework to examine data-related challenges and opportunities in digital health design.

Development and Formative Evaluation of a Virtual Exercise Platform for a Community Fitness Center Serving Individuals With Physical Disabilities: Mixed Methods Study

BACKGROUND

People with disabilities experience numerous barriers to being physically active, such as transportation issues, a lack of trained exercise professionals who understand disabilities, and facility access. The use of a virtual exercise platform (VEP) may provide an alternative and limit certain barriers.

OBJECTIVE

The aim of this mixed method study was to evaluate user interaction (effectiveness, efficiency, and satisfaction), the strengths and weaknesses of the user interface, and the user experience with a VEP.

METHODS

Participants were recruited from a community fitness facility that offers programs for people with disabilities. Inclusion criteria were being older than 18 years, fluent in English, and availability of internet access. Features of the VEP included articles, prerecorded videos, live Zoom classes, web-based class registration, weekly progress tracking, incentives, and surveys. A one-on-one Zoom session was scheduled with each participant, during which they completed certain tasks: (1) create an account or login, (2) register for class, (3) join class, (4) add to calendar, and (5) complete surveys. As participants completed tasks, quantitative observations (time on task, task success, rate of task completion, and number of errors by users, which determined task difficulty), qualitative observations were made and interviews were conducted at the end of the session. The "concurrent think-aloud" method was encouraged by the moderator to gauge participants' thoughts as they worked through testing. Participants also completed the System Usability Scale (SUS) and Questionnaire for User Interface Satisfaction (QUIS).

RESULTS

A total of 5 people with disabilities (3 male, 2 female), aged 36-78 (mean 54) years, with education levels from high school to PhD, were recruited. Devices used for testing included a laptop (n=3), a Chromebook (n=1), and a desktop (n=1). All participants completed tasks #1 and #2 without errors but could not complete task #4. One participant completed task #5 with difficulty and another completed task #3 with difficulty. The average time to complete each task was: (1) 82 seconds (55-110), (2) 11 seconds (4-21), (3) 9 seconds (5-27), and (4) 921.5 seconds (840-958). The mean SUS score was 86.5/100, and the mean user QUIS score was 8.08 out of 10. Qualitative observations indicated that the system was simple, user-friendly, and accessible.

CONCLUSIONS

People with disabilities reported high usability and user satisfaction with the web-based exercise platform, and the system appears to be an efficient and effective physical activity option.

The usage of data in NHS primary care commissioning: a realist evaluation

BACKGROUND

To improve health outcomes and address mounting costs pressures, policy-makers have encouraged primary care commissioners in the British National Health Service (NHS) to increase the usage of data in decision-making. However, there exists limited research on this topic. In this study, we aimed to understand how and why primary care commissioners use data (i.e. quantitative, statistical information) to inform commissioning, and what outcomes this leads to.

METHODS

A realist evaluation was completed to create context-mechanism-outcome configurations (CMOs) relating to the contexts influencing the usage of data in primary care commissioning. Using a realist logic of analysis and drawing on substantive theories, we analysed qualitative content from 30 interviews and 51 meetings (51 recordings and 19 accompanying meeting minutes) to develop CMOs. Purposive sampling was used to recruit interviewees from diverse backgrounds.

RESULTS

Thirty-five CMOs were formed, resulting in an overarching realist programme theory. Thirteen CMOs were identical and 3 were truncated versions of those formed in an existing realist synthesis on the same topic. Seven entirely new CMOs, and 12 refined and enhanced CMOs vis-à-vis the synthesis were created. The findings included CMOs containing contexts which facilitated the usage of data, including the presence of a data champion and commissioners' perceptions that external providers offered new skillsets and types of data. Other CMOs included contexts presenting barriers to using data, such as data not being presented in an interoperable way with consistent definitions, or financial pressures inhibiting commissioners' abilities to make evidence-based decisions.

CONCLUSIONS

Commissioners are enthusiastic about using data as a source of information, a tool to stimulate improvements, and a warrant for decision-making. However, they also face considerable challenges when using them. There are replicable contexts available to facilitate commissioners' usage of data, which we used to inform policy recommendations. The findings of this study and our recommendations are pertinent in light of governments' increasing commitment to data-driven commissioning and health policy-making.

Outcomes of End-User Testing of a Care Coordination Mobile App With Families of Children With Special Health Care Needs: Simulation Study

BACKGROUND

Care for children with special health care needs relies on a network of providers who work to address the medical, behavioral, developmental, educational, social, and economic needs of the child and their family. Family-directed, manually created visual depictions of care team composition (ie, care mapping) and detailed note-taking curated by caregivers (eg, care binders) have been shown to enhance care coordination for families of these children, but they are difficult to implement in clinical settings owing to a lack of integration with electronic health records and limited visibility of family-generated insights for care providers. Caremap is an electronic health record-integrated digital personal health record mobile app designed to integrate the benefits of care mapping and care binders. Currently, there is sparse literature describing end-user participation in the co-design of digital health tools. In this paper, we describe a project that evaluated the usability and proof of concept of the Caremap app through end-user simulation.

OBJECTIVE

This study aimed to conduct proof-of-concept testing of the Caremap app to coordinate care for children with special health care needs and explore early end-user engagement in simulation testing. The specific aims included engaging end users in app co-design via app simulation, evaluating the usability of the app using validated measures, and exploring user perspectives on how to make further improvements to the app.

METHODS

Caregivers of children with special health care needs were recruited to participate in a simulation exercise using Caremap to coordinate care for a simulated case of a child with complex medical and behavioral needs. Participants completed a postsimulation questionnaire adapted from 2 validated surveys: the Pediatric Integrated Care Survey (PICS) and the user version of the Mobile Application Rating Scale (uMARS). A key informant interview was also conducted with a liaison to Spanish-speaking families regarding app accessibility for non-English-speaking users.

RESULTS

A Caremap simulation was successfully developed in partnership with families of children with special health care needs. Overall, 38 families recruited from 19 different US states participated in the simulation exercise and completed the survey. The average rating for the survey adapted from the PICS was 4.1 (SD 0.82) out of 5, and the average rating for the adapted uMARS survey was 4 (SD 0.83) out of 5. The highest-rated app feature was the ability to track progress toward short-term, patient- and family-defined care goals.

CONCLUSIONS

Internet-based simulation successfully facilitated end-user engagement and feedback for a digital health care coordination app for families of children with special health care needs. The families who completed simulation with Caremap rated it highly across several domains related to care coordination. The simulation study results elucidated key areas for improvement that translated into actionable next steps in app development.

Clinical research staff perceptions on a natural language processing-driven tool for eligibility prescreening: An iterative usability assessment

BACKGROUND

Participant recruitment is a barrier to successful clinical research. One strategy to improve recruitment is to conduct eligibility prescreening, a resource-intensive process where clinical research staff manually reviews electronic health records data to identify potentially eligible patients. Criteria2Query (C2Q) was developed to address this problem by capitalizing on natural language processing to generate queries to identify eligible participants from clinical databases semi-autonomously.

OBJECTIVE

We examined the clinical research staff's perceived usability of C2Q for clinical research eligibility prescreening.

METHODS

Twenty clinical research staff evaluated the usability of C2Q using a cognitive walkthrough with a think-aloud protocol and a Post-Study System Usability Questionnaire. On-screen activity and audio were recorded and transcribed. After every-five evaluators completed an evaluation, usability problems were rated by informatics experts and prioritized for system refinement. There were four iterations of system refinement based on the evaluation feedback. Guided by the Organizational Framework for Intuitive Human-computer Interaction, we performed a directed deductive content analysis of the verbatim transcriptions.

RESULTS

Evaluators aged from 24 to 46 years old (33.8; SD: 7.32) demonstrated high computer literacy (6.36; SD:0.17); female (75 %), White (35 %), and clinical research coordinators (45 %). C2Q demonstrated high usability during the final cycle (2.26 out of 7 [lower scores are better], SD: 0.74). The number of unique usability issues decreased after each refinement. Fourteen subthemes emerged from three themes: seeking user goals, performing well-learned tasks, and determining what to do next.

CONCLUSIONS

The cognitive walkthrough with a think-aloud protocol informed iterative system refinement and demonstrated the usability of C2Q by clinical research staff. Key recommendations for system development and implementation include improving system intuitiveness and overall user experience through comprehensive consideration of user needs and requirements for task completion.

The Acceptability of Using Augmented Reality as a Mechanism to Engage Children in Asthma Inhaler Technique Training: Qualitative Interview Study With Deductive Thematic Analysis

BACKGROUND

Inhaled medications or inhalers provide first-line pharmacotherapeutic treatment for patients with asthma for both acute symptomatic relief and long-term management to keep symptoms under control. A good technique requires only basic instruction and training; however, a recent study identified that 92% of children do not follow all correct steps when using inhalers. There is a growing interest in technology-enhanced asthma education, with evidence demonstrating improvements in knowledge and treatment adherence. Subsequently, there are calls to explore the role of technology-based solutions in improving asthma management and disease outcomes from public health experts, health professionals, and patients with asthma. Augmented reality (AR) technology is an information delivery mechanism with proven efficacy in educational settings. AR displays digital content in a real-world environment using the camera on a smartphone or tablet device to create an immersive learning experience.

OBJECTIVE

The study aimed to evaluate the acceptability of AR as a mechanism for delivering asthma inhaler technique education from the perspective of children with asthma and their parents and health professionals, examined through the theoretical framework of acceptability (TFA).

METHODS

An asthma education resource enhanced with AR technology was created to provide inhaler technique education to children. An iterative co-design process was undertaken with target end users for a qualitative evaluation. The participants were 8 to 12 years old with asthma, their caregivers, and health professionals who had experience in managing asthma. Qualitative data were obtained through semistructured one-on-one interviews. Deductive thematic analysis using TFA was undertaken using NVivo software 2020 to assess the acceptability of AR as a delivery modality for asthma inhaler technique education.

RESULTS

Overall, 6 health care professionals, 5 asthmatic children, and 5 caregivers of children with asthma totaled a sample of 16. The use of AR in the asthma inhaler resource was found to be acceptable when responses were examined in accordance with TFA. Each of the 7 component constructs of TFA was coded throughout the 16 interviews, with perceived effectiveness (157 times) and affective attitude (63 times) coded most frequently. Positive responses included the intervention being accessible, easy to use, interesting, and fitting within the users' value systems. Negative responses included the need to maintain an interest in children and concerns about the loss of face-to-face interaction with health professionals.

CONCLUSIONS

AR appears to be an acceptable modality for delivering asthma education to children when explored using TFA constructs. Although some challenges were identified with the use of AR, the results were predominantly positive. Future designs of asthma education interventions involving AR should consider the results of this study, and further research should focus on the feasibility, usability, and barriers and facilitators of behavior change to ensure the successful implementation and uptake of AR into clinical settings.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.1177/16094069211042229.

A framework for design and usability testing of telerehabilitation system for adults with chronic diseases: A panoramic scoping review

Objective

This scoping review aimed to identify the design and usability testing of a telerehabilitation (TR) system, and its characteristics and functionalities that are best-suited for rehabilitating adults with chronic diseases.

Methods

Searches were conducted in PubMed, EBSCO, Web of Science, and Cochrane library for studies published between January 2017 and December 2022. We followed the Joanna Briggs Institute guidelines and the framework by Arksey and O'Malley. Screening was undertaken by two reviewers, and data extraction was undertaken by the first author. Then, the data were further reviewed and discussed thoroughly with the team members.

Results

A total of 31 results were identified, with the core criteria of developing and testing a telerehabilitation system, including a mobile app for cardiovascular diseases, cancer, diabetes, and chronic respiratory disorders. All developed systems resulted from multidisciplinary teams and employed mixed-methods research. We proposed the "input-process-output" framework that identified phases of both system design and usability testing. Through system design, we reported the use of user-centered design, iterative design, users' needs and characteristics, theory underpinning development, and the expert panel in 64%, 75%, 86%, 82%, and 71% of the studies, respectively. We recorded the application of moderated usability testing, unmoderated testing (1), and unmoderated testing (2) in 74%, 63%, and 15% of the studies, respectively. The identified design and testing activities produced a matured system, a high-fidelity prototype, and a released system in 81.5%, 15%, and 3.5%, respectively.

Conclusion

This review provides a framework for TR system design and testing for a wide range of chronic diseases that require prolonged management through remote monitoring using a mobile app. The identified "input-process-output" framework highlights the inputs, design, development, and improvement as components of the system design. It also identifies the "moderated-unmoderated" model for conducting usability testing. This review illustrates characteristics and functionalities of the TR systems and healthcare professional roles.

Usability evaluation of mHealth apps for elderly individuals: a scoping review

BACKGROUND

Usability is a key factor affecting the acceptance of mobile health applications (mHealth apps) for elderly individuals, but traditional usability evaluation methods may not be suitable for use in this population because of aging barriers. The objectives of this study were to identify, explore, and summarize the current state of the literature on the usability evaluation of mHealth apps for older adults and to incorporate these methods into the appropriate evaluation stage.

METHODS

Electronic searches were conducted in 10 databases. Inclusion criteria were articles focused on the usability evaluation of mHealth apps designed for older adults. The included studies were classified according to the mHealth app usability evaluation framework, and the suitability of evaluation methods for use among the elderly was analyzed.

RESULTS

Ninety-six articles met the inclusion criteria. Research activity increased steeply after 2013 (n = 92). Satisfaction (n = 74) and learnability (n = 60) were the most frequently evaluated critical measures, while memorability (n = 13) was the least evaluated. The ratios of satisfaction, learnability, operability, and understandability measures were significantly related to the different stages of evaluation (P < 0.05). The methods used for usability evaluation were questionnaire (n = 68), interview (n = 36), concurrent thinking aloud (n = 25), performance metrics (n = 25), behavioral observation log (n = 14), screen recording (n = 3), eye tracking (n = 1), retrospective thinking aloud (n = 1), and feedback log (n = 1). Thirty-two studies developed their own evaluation tool to assess unique design features for elderly individuals.

CONCLUSION

In the past five years, the number of studies in the field of usability evaluation of mHealth apps for the elderly has increased rapidly. The mHealth apps are often used as an auxiliary means of self-management to help the elderly manage their wellness and disease. According to the three stages of the mHealth app usability evaluation framework, the critical measures and evaluation methods are inconsistent. Future research should focus on selecting specific critical measures relevant to aging characteristics and adapting usability evaluation methods to elderly individuals by improving traditional tools, introducing automated evaluation tools and optimizing evaluation processes.

Design of Smart Head–Mounted Display Technology: A Convergent Mixed-Methods Study

Introduction: The purpose of this study was to characterize functional impairments and human factor considerations that affect perceptions and preferences for head-mounted display (HMD) technology for adults with low vision and chronic eye disease. Methods: Through a convergent mixed-methods design, participants with visual impairments (age-related macular degeneration, diabetic retinopathy, glaucoma, or retinitis pigmentosa) were recruited. Participants completed the Impact of Vision Impairment (IVI) questionnaire, used commercially available HMDs (eSight, NuEyes, and Epson Moverio), and were interviewed. The IVI was used to identify groups with low, moderate, and high vision–related quality of life (VRQOL). Transcribed interviews were analyzed using a thematic approach. The survey and qualitative findings were integrated using mixed-methods joint display analysis. Results: Twenty-one participants were enrolled (mean age of 58.2 years, 57% male, median Snellen acuity of 20/40 [range: 20/20–hand movement]). An equal number ( n = 9) expressed a preference for eSight and NuEyes, while ( n = 3) preferred the Moverio. Participants emphasized ease of use, including HMD controls and screen, as common reasons for preference. Those with lower IVI well-being scores preferred eSight due to vision improvement. Those with moderate IVI well-being scores preferred NuEyes due to comfort and size. Those with high IVI well-being scores cited usability as the most important feature. Discussion: User preferences for HMD features were associated with VRQOL. A mixed-methods approach explained how varying degrees of visual impairment and HMD preferences were qualitatively related to usability at the individual level. Implications for Practitioners: To increase acceptance, new HMD development for low vision should focus on performance, usability, and human factors engineering. Although HMD technology can benefit individuals with low vision, device features and functions vary in meaningful ways based on vision parameters. Practitioners should be aware of how patient and device variations influence preferences when they recommend wearable systems and optimize training to harness these systems.

The Development of a Hypertension Prevention and Financial Incentive Mobile Health Program Using a “No-Code” Mobile App Builder: A Development and Usability Study (Preprint)

BACKGROUND

Regular physical activity (PA) is a key lifestyle component for hypertension prevention. Previous studies have shown that mobile health (mHealth) apps can be an effective tool for improving PA behaviors. However, adherence to and poor engagement with these apps is a challenge. A potential solution to overcome this challenge may be to combine financial incentives with innovative behavior theory, such as the Multiprocess Action Control (M-PAC) framework. Currently, there is a lack of PA financial incentive-driven M-PAC mHealth programs aimed at hypertension prevention.

OBJECTIVE

We aimed to describe the process of developing an 8-week mHealth PA and financial-incentive hypertension education program (Healthy Hearts) and to evaluate usability of the Healthy Hearts program.

METHODS

The first 2 stages of the Integrate, Design, Assess, and Share framework were used to guide the development of the Healthy Hearts program. The development process consisted of 2 phases. In phase 1, the research team met to discuss implementing the M-PAC framework to adopt an existing web-based hypertension prevention program to a mobile app. The app was developed using a no-code app development platform, Pathverse (Pathverse Inc), to help decrease overall development time. In phase 2, we created a prototype and conducted usability testing to evaluate lesson 1 of the Healthy Hearts program to further enhance the user experience. We used semistructured interviews and the mHealth App Usability Questionnaire to evaluate program acceptability and usability.

RESULTS

Intervention development among the research team successfully created an 8-week financial-incentive hypertension education program for adults aged 40-65 years who did not currently meet the Canadian Physical Activity Guidelines (<150 minutes of moderate to vigorous PA per week). This program lasted 8 weeks and comprised 25 lessons guided by the M-PAC framework. The program used various behavior change techniques to further support PA adherence. Usability testing of the first lesson was successful, with 6 participants recruited for 2 rounds of testing. Feedback was gathered to enhance the content, layout, and design of the Healthy Hearts program to prepare the mHealth program for feasibility testing. Results of round 1 of usability testing suggested that the content delivered in the lessons was long. Therefore, the content was divided into multiple lessons before round 2 of usability testing, where feedback was only on design preferences. A minimum viable product was created with these results.

CONCLUSIONS

The iterative development process and the usability assessments suggested by the Integrate, Design, Assess, and Share framework enabled participants to provide valuable feedback on the content, design, and layout of the program before advancing to feasibility testing. Furthermore, the use of the "no-code" app development tool enabled our team to rapidly make changes to the app based on user feedback during the iterative design process.

Exploring the feasibility of a network of organizations for pain rehabilitation: What are the lessons learned?

Background and aims

Integration of care is lacking for chronic musculoskeletal pain patients. Network Pain Rehabilitation Limburg, a transmural health care network, has been designed to provide integrated rehabilitation care from a biopsychosocial perspective to improve patients’ levels of functioning. This feasibility study aims to provide insight into barriers and facilitators for the development, implementation, and transferability.

Methods

This study was conducted with a three-phase iterative and incremental design from October 2017 to October 2018. The network comprises two rehabilitation practices, and three local primary care networks, with a general practitioner together with, a mental health practice nurse, and a physiotherapist or exercise therapist. These stakeholders with a random sample of participating patients took part in evaluations, consisting of interviews, focus groups, and observations. Field notes and observations were recorded during meetings. The Consolidated Framework for Implementation Research guided data collection and analysis. Results were used to refine the next phase.

Results

According to health care professionals, guidelines and treatment protocols facilitate consistency and transparency in collaboration, biopsychosocial language, and treatment. One mentioned barrier is the stigmatization of chronic pain by the general population. In regular care, approaches are often more biomedical than biopsychosocial, causing patients to resist participating. The current organization of health care acts as a barrier, complicating implementation between and within practices. Health care professionals were enthusiastic about the iterative, bottom-up development. A critical mass of participating organizations is needed for proper implementation.

Conclusion

Network Pain Rehabilitation Limburg is feasible in daily practice if barriers are overcome and facilitators of development, implementation, and transferability are promoted. These findings will be used to refine Network Pain Rehabilitation Limburg. A large-scale process and effect evaluation will be performed. Our implementation strategies and results may assist other health care organizations aspiring to implement a transmural network using a similar model.

Trail registration

Registration number: NTR6654 or https://trialsearch.who.int/Trial2.aspx?TrialID=NTR6654.

Assessing adherence to and usability of Experience Sampling Method (ESM) and actigraph in patients with Schizophrenia Spectrum Disorder: A mixed-method study

The application of innovative technologies in psychiatry is promising, but the debate about its feasibility is not settled. Our aim was to investigate and compare adherence to and usability of 7-day monitoring with an Experience Sampling Method (ESM) and Actigraphy among a sample of individuals with a diagnosis of Schizophrenia Spectrum Disorders (SSD) and paired healthy controls. Twenty-two patients living in residential facilities (RFs), 20 outpatients and 26 controls were enrolled in this study as part of the DiAPAson project. Participants wore an actigraph and were evaluated for daily time use and mood with a smartphone-based ESM. Then, they completed questionnaires to assess the usability of the devices and were interviewed. Adherence was assessed as the percentage of time spent wearing actigraph and answering ESM notifications. Residential patients, compared to controls, showed significantly higher usability of the actigraph and lower adherence to both the actigraph and ESM. From the qualitative interviews, four high-order themes emerged: effects of monitoring, adherence and usability, emotions and mental states, and advice. Findings are promising for the application of these methodologies in such populations. The role of multidisciplinary staff in RFs is crucial for guaranteeing the realization of such projects.

An mHealth App to Promote Adherence to Immunosuppressant Medication and Track Symptoms in Children After Hematopoietic Stem Cell Transplant: Protocol for a Mixed Methods Usability Study

Background

In the United States, poor adherence accounts for up to 70% of all medication-related hospital admissions, resulting in $100 billion in health care costs annually. In pediatrics, adherence is largely dependent on caregivers. In a high-risk hematopoietic stem cell transplant (HSCT) population, caregivers are isolated with their child due to infection risk and must manage challenging treatment regimens at home, often with limited time and support. Complex behavioral interventions, typically employed to address adherence, are difficult to deliver and manage in the context of these daily tasks. The most successful adherence interventions, and thus improved clinical outcomes, have included mobile health (mHealth) reminder approaches and a direct measure of adherence.

Objective

This is a 3-phase project, with this protocol describing phase 2, to determine the usability and feasibility of an mHealth app (BMT4me) designed to promote adherence to immunosuppressant medication and to track symptoms among children who received HSCT.

Methods

This study uses an iterative convergent mixed methods design to develop and assess the usability and feasibility of an adherence digital health intervention. We will recruit 15 caregivers of pediatric patients receiving HSCT to complete user testing. Qualitative and quantitative data will be integrated to enhance and expand upon study findings.

Results

Enrollment began in September 2021 and is ongoing. A total of 7 caregivers have enrolled. We anticipate completion by fall 2022. We anticipate high usability scores and a better understanding of unique features within the app that are needed for HSCT families post transplant. To date, usability scores among enrolled participants are greater than 70%. Feedback from qualitative interviews is being used to further adapt the app by adding specific weekly logs, call provider options, and voice to text.

Conclusions

This protocol describes a mixed methods usability and feasibility study to develop and implement a smartphone app for caregivers of children receiving HSCT. The app was designed to improve immunosuppressant adherence and to track symptoms in the acute phase post discharge. Study findings will inform further refinement of the app and the feasibility of a pilot randomized controlled trial examining efficacy on clinical outcomes.

Trial Registration

ClinicalTrials.gov NCT04976933; https://clinicaltrials.gov/ct2/show/NCT04976933

International Registered Report Identifier (IRRID)

DERR1-10.2196/39098

Process evaluation of the New Interventions for independence in Dementia Study (NIDUS) Family stream randomised controlled trial: protocol

INTRODUCTION

New Interventions for independence in Dementia Study (NIDUS)-Family is an Alzheimer's Society funded new manualised, multimodal psychosocial intervention to support people living with dementia (PLWD) to achieve goals that they and their family carers set, towards living as independently and as well as possible at home for longer. This process evaluation will be embedded within the NIDUS-Family Randomised Controlled Trial intervention-arm (n=199), testing how the intervention influences change, as measured by goal attainment. The evaluation will test, refine and develop the NIDUS-Family theoretical model, associated causal assumptions and logic model to identify key mechanisms of impact, implementation and contextual factors influencing the intervention's effectiveness. Findings will inform how the programme is implemented in practice.

METHODS AND ANALYSIS

The process evaluation will be theory driven and apply a convergent mixed-methods design. Dyads (PLWD and family carer) will be purposively sampled based on high or low Goal Attainment Scaling scores (trial primary outcome). Qualitative interviews with dyads (approx. n=30) and their respective facilitators post-trial will explore their experiences of receiving and delivering the intervention. Interviews will be iteratively thematically analysed. Matching observational quantitative data will be collected concurrently from videorecordings and/or audiorecordings of NIDUS-Family dyad trial sessions. Further quantitative data will be collected through an acceptability questionnaire for all intervention-arm dyads (n=199). Mixed-method integration will use an interactive analysis strategy, considering qualitative and quantitative findings through mixed-method matrix for dyadic level 'case studies', and a joint display for 'population' level analysis and interpretation.

ETHICS AND DISSEMINATION

Ethical approval was received from Camden & Kings Cross Research Ethics Committee (REC). Study reference: 19/LO/1667. IRAS project ID: 271 363. This work is carried out within the UCL Alzheimer's Society Centre of Excellence (grant 300) for Independence at home, NIDUS programme.Findings will be disseminated through publications and conferences, and as recommendations for the implementation study and strategy.

TRIAL REGISTRATION NUMBER

ISRCTN11425138.

A WeChat-Based Rehabilitation Platform for Children and Adolescents with Congenital Heart Disease to Promote Cardiac FITness (HeartFIT): Protocol for a Mixed-Methods Strategy from Evidence-Based Design to Pilot Study

Progress in medical and surgical care has tremendously improved the survival rates of children with congenital heart disease (CHD). However, reduced aerobic capacity and health-related issues remain a threaten to quality survival and prevention of related complications among children and adolescents with CHD. This research program aims to develop and evaluate a WeChat-based health platform (HeartFIT) to facilitate cardiac rehabilitation and promote physical fitness for this rapidly expanding young population. The study protocol describes the use of an iterative process of using a mixed-methods strategy to develop, refine, and pilot test the proposed HeartFIT platform. A sequential problem-solving process comprising four iterative phases with ongoing end-user input will be implemented. In phase 1, relevant literature was systematically reviewed (completed) and then child-parent dyads will be interviewed to understand the broad context and the requirements and considerations of the target population toward the WeChat-based rehabilitation platform. In phase 2, key features and priority functionalities for the platform will be ideated and refined, and a digital interactive prototype will be created. In phase 3, heuristic evaluation and three rounds of end-user testing will be conducted to ensure further refinement and usability of the prototype. In phase 4, a prospective pilot study will be performed to investigate the feasibility, acceptability, and preliminary efficacy of the developed platform over a 12-week intervention period. If HeartFIT intervention is feasible, acceptable, and demonstrates promising efficacy, an adequately powered randomized controlled trial (future work) will be deployed to test the real-world effectiveness of the intervention.

Building Bridges for "Palliative Care-in-Place": Development of a mHealth Intervention for Informal Home Care

Background

In Palliative Care (PC), family and close people are an essential part of provision of care. They assume highly complex tasks for which they are not prepared, with considerable physical, psychological, social and economic impact. Informal Caregivers (ICs) often falter in the final stage of life and develop distress, enhancing emotional burden and complicated grief. The lack of available and accessible in-person counselling resources is often reported by ICs. Online resources can promote early access to help and support for patient-IC dyads in palliative care. The primary aim of this research is to co-design, develop and test the feasibility of the Help2Care-PAL mHealth app that addresses the needs of ICs of palliative patients cared for at home. This Digital Health Intervention (DHI) in palliative care will be used for education, symptom management, communication and decision-making, to enhance Quality of Life (QoL) of patients and ICs, fostering anticipatory grief and the reach and efficiency of services.

Methods

This study will use an iterative co-design process and convergent mixed-methods design, following the MORECare consensus for developing a complex intervention. Construction of the DHI will follow four main phases: (I) a needs assessment (a cross-sectional survey, individual interviews with ICs and focus groups with professionals with community palliative care experience); (II) design and co-production of mHealth materials and interventions to support ICs; (III) the development of a mHealth app; and (IV) usability and feasibility of the mHealth app. The Help2Care-PAL platform seeks to build resources from the perspectives and needs of both family dyads and nursing professionals working in the field of community palliative care. User-centeredness will be ensured by the active participation of patient-IC dyads and professionals of the palliative care community.

Discussion

This mixed-method study will offer new insights on needs and expectations of patient-IC dyads and nurses in community palliative care regarding caregiving preparedness and online health resources. Through the implementation of an adaptive digital tool, we aim to improve access to palliative care family support, which is highly linked with the wellbeing of patients and especially new ICs.

Usability Testing of a Digital Assessment Routing Tool for Musculoskeletal Disorders: An Iterative Convergent Mixed Methods Study (Preprint)

Background

Musculoskeletal disorders negatively affect millions of patients worldwide, placing significant demand on health care systems. Digital technologies that improve clinical outcomes and efficiency across the care pathway are development priorities. We developed the musculoskeletal Digital Assessment Routing Tool (DART) to enable self-assessment and immediate direction to the right care.

Objective

We aimed to assess and resolve all serious DART usability issues to create a positive user experience and enhance system adoption before conducting randomized controlled trials for the integration of DART into musculoskeletal management pathways.

Methods

An iterative, convergent mixed methods design was used, with 22 adult participants assessing 50 different clinical presentations over 5 testing rounds across 4 DART iterations. Participants were recruited using purposive sampling, with quotas for age, habitual internet use, and English-language ability. Quantitative data collection was defined by the constructs within the International Organization for Standardization 9241-210-2019 standard, with user satisfaction measured by the System Usability Scale. Study end points were resolution of all grade 1 and 2 usability problems and a mean System Usability Scale score of ≥80 across a minimum of 3 user group sessions.

Results

All participants (mean age 48.6, SD 15.2; range 20-77 years) completed the study. Every assessment resulted in a recommendation with no DART system errors and a mean completion time of 5.2 (SD 4.44, range 1-18) minutes. Usability problems were reduced from 12 to 0, with trust and intention to act improving during the study. The relationship between eHealth literacy and age, as explored with a scatter plot and calculation of the Pearson correlation coefficient, was performed for all participants (r=–0.2; 20/22, 91%) and repeated with a potential outlier removed (r=–0.23), with no meaningful relationships observed or found for either. The mean satisfaction for daily internet users was highest (19/22, 86%; mean 86.5, SD 4.48; 90% confidence level [CL] 1.78 or –1.78), with nonnative English speakers (6/22, 27%; mean 78.1, SD 4.60; 90% CL 3.79 or –3.79) and infrequent internet users scoring the lowest (3/22, 14%; mean 70.8, SD 5.44; 90% CL 9.17 or –9.17), although the CIs overlap. The mean score across all groups was 84.3 (SD 4.67), corresponding to an excellent system, with qualitative data from all participants confirming that DART was simple to use.

Conclusions

All serious DART usability issues were resolved, and a good level of satisfaction, trust, and willingness to act on the DART recommendation was demonstrated, thus allowing progression to randomized controlled trials that assess safety and effectiveness against usual care comparators. The iterative, convergent mixed methods design proved highly effective in fully evaluating DART from a user perspective and could provide a blueprint for other researchers of mobile health systems.

International Registered Report Identifier (IRRID)

RR2-10.2196/27205

Usability Testing of a Patient-Centered Mobile Health App for Supporting and Guiding the Pediatric Emergency Department Patient Journey: Mixed Methods Study

BACKGROUND

Patient experience in emergency departments (EDs) remains often suboptimal and can be a source of stress, particularly in pediatric settings. In an attempt to support patients and their families before, during, and after their visit to a pediatric ED, a mobile health (mHealth) app was developed by a multidisciplinary team based on patient-centered care principles.

OBJECTIVE

This study aims to evaluate the usability (effectiveness, efficiency, and satisfaction) of a new mHealth app, InfoKids, by potential end users through usability testing.

METHODS

The app was assessed through an in-laboratory, video-recorded evaluation in which participants had to execute 9 goal-oriented tasks, ranging from account creation to the reception of a diagnostic sheet at the end of the emergency care episode. Effectiveness was measured based on the task completion rate, efficiency on time on task, and user satisfaction according to answers to the System Usability Scale questionnaire. Think-aloud usability sessions were also transcribed and analyzed. Usability problems were rated for their severity and categorized according to ergonomic criteria.

RESULTS

A total of 17 parents participated in the study. The overall completion rate was 97.4% (149/153). Overall, they reported good effectiveness, with the task successfully completed in 88.2% (135/153) of cases (95% CI 83%-93%). Each task, with the exception of the first, created difficulties for some participants but did not prevent their completion by most participants. Users reported an overall good to excellent perceived usability of the app. However, ergonomic evaluation identified 14 usability problems occurring 81 time. Among these, 50% (7/14) were serious as their severity was rated as either major or catastrophic and indicated areas of improvements for the app. Following the suggested usability improvements by participants, mitigation measures were listed to further improve the app and avoid barriers to its adoption.

CONCLUSIONS

Usability of the InfoKids app was evaluated as good to excellent by users. Areas of improvement were identified, and mitigation measures were proposed to inform its development toward a universal app for all ED patients visiting a digitalized institution. Its contribution could also be useful in paving the way for further research on mobile apps aimed at supporting and accompanying patients in their care episodes, as research in this area is scarce.

Adapting an evidence-based e-learning CBT program into a mobile app for people experiencing gambling-related problems: A Formative Study

Background

Many people who experience harm and problems from gambling do not seek treatment from gambling treatment services because of personal and resource barriers. Mobile health (mHealth) interventions are widely used across diverse health care areas and populations. However, there are few in the gambling harm field, despite their potential as an additional modality for delivering treatment and support.

Objective

This study aims to understand the needs, preferences, and priorities of people experiencing gambling harms and who are potential end users of a cognitive behavioral therapy mHealth intervention to inform design, features, and functions.

Methods

Drawing on a mixed methods approach, we used creators and domain experts to review the GAMBLINGLESS web-based program and convert it into an mHealth prototype. Each module was reviewed against the original evidence base to maintain its intended fidelity and conceptual integrity. Early wireframes, design ideas (look, feel, and function), and content examples were developed to initiate discussions with end users. Using a cocreation process with a young adult, a Māori, and a Pasifika peoples group, all with experiences of problem or harmful gambling, we undertook 6 focus groups: 2 cycles per group. In each focus group, participants identified preferences, features, and functions for inclusion in the final design and content of the mHealth intervention.

Results

Over 3 months, the GAMBLINGLESS web-based intervention was reviewed and remapped from 4 modules to 6. This revised program is based on the principles underpinning the transtheoretical model, in which it is recognized that some end users will be more ready to change than others. Change is a process that unfolds over time, and a nonlinear progression is common. Different intervention pathways were identified to reflect the end users’ stage of change. In all, 2 cycles of focus groups were then conducted, with 30 unique participants (13 Māori, 9 Pasifika, and 8 young adults) in the first session and 18 participants (7 Māori, 6 Pasifika, and 5 young adults) in the second session. Prototype examples demonstrably reflected the focus group discussions and ideas, and the features, functions, and designs of the Manaaki app were finalized. Attributes such as personalization, cultural relevance, and positive framing were identified as the key. Congruence of the final app attributes with the conceptual frameworks of the original program was also confirmed.

Conclusions

Those who experience gambling harms may not seek help. Developing and demonstrating the effectiveness of new modalities to provide treatment and support are required. mHealth has the potential to deliver interventions directly to the end user. Weaving the underpinning theory and existing evidence of effective treatment with end-user input into the design and development of mHealth interventions does not guarantee success. However, it provides a foundation for framing the intervention’s mechanism, context, and content, and arguably provides a greater chance of demonstrating effectiveness.

HEARTSMAP-U: Adapting a Psychosocial Self-Screening and Resource Navigation Support Tool for Use by Post-secondary Students

BACKGROUND

Mental health challenges are highly prevalent in the post-secondary educational setting. Screening instruments have been shown to improve early detection and intervention. However, these tools often focus on specific diagnosable conditions, are not always designed with students in mind, and lack resource navigational support.

OBJECTIVE

The aim of this study was to describe the adaptation of existing psychosocial assessment (HEARTSMAP) tools into a version that is fit-for-purpose for post-secondary students, called HEARTSMAP-U.

METHODS

We underwent a three-phase, multi-method tool adaptation process. First, a diverse study team proposed a preliminary version of HEARTSMAP-U and its conceptual framework. Second, we conducted a cross-sectional expert review study with Canadian mental health professionals (N = 28), to evaluate the clinical validity of tool content. Third, we conducted an iterative series of six focus groups with diverse post-secondary students (N = 54), to refine tool content and language, and ensure comprehensibility and relevance to end-users.

RESULTS

The adaptation process resulted in the HEARTSMAP-U self-assessment and resource navigational support tool, which evaluates psychosocial challenges across 10 sections. In Phase two, clinician experts expressed that HEARTSMAP-U's content aligned with their own professional experiences working with students. In Phase three, students identified multiple opportunities to improve the tool's end-user relevance by calling for more "common language," such as including examples, definitions, and avoiding technical jargon.

CONCLUSIONS

The HEARTSMAP-U tool is well-positioned for further studies of its quantitative psychometric properties and clinical utility in the post-secondary educational setting.

Towards Understanding the Usability Attributes of AI-Enabled eHealth Mobile Applications

Mobile application (app) use is increasingly becoming an essential part of our daily lives. Due to their significant usefulness, people rely on them to perform multiple tasks seamlessly in almost all aspects of everyday life. Similarly, there has been immense progress in artificial intelligence (AI) technology, especially deep learning, computer vision, natural language processing, and robotics. These technologies are now actively being implemented in smartphone apps and healthcare, providing multiple healthcare services. However, several factors affect the usefulness of mobile healthcare apps, and usability is an important one. There are various healthcare apps developed for each specific task, and the success of these apps depends on their performance. This study presents a systematic review of the existing apps and discusses their usability attributes. It highlights the usability models, outlines, and guidelines proposed in previous research for designing apps with improved usability characteristics. Thirty-nine research articles were reviewed and examined to identify the usability attributes, framework, and app design conducted. The results showed that satisfaction, efficiency, and learnability are the most important usability attributes to consider when designing eHealth mobile apps. Surprisingly, other significant attributes for healthcare apps, such as privacy and security, were not among the most indicated attributes in the studies.

Effectiveness of mobile health-based self-management application for posttransplant cares: A systematic review

BACKGROUND AND AIMS

Patients after transplantation need medical management for the rest of their lives, and self-management seems to lead to greater adherence to medical standards, improve early physical changes, and increase patient empowerment. The main objective of this article is to systematic review of the consideration to mobile health applications (m-Health apps) used in transplantation.

METHODS

A systematic search was conducted MEDLINE (through PubMed), Web of Science, Scopus, and Science Direct from inception to November 2020. The Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) statement was used in this study. Comprehensive research was carried out using a combination of keywords and MeSH terms associated with m-Health, empowerment, self-management, and transplantation. Two independent reviewers screened titles and abstracts, assessed full-text articles, and extracted data from articles that met inclusion criteria. Eligible studies were original research articles that included posttransplant care and mobile phone-based applications to support self-management and self-care. Also, thesis, book chapters, letters to editors, short briefs, reports, technical reports, book reviews, systematic reviews, or meta-analysis were excluded.

RESULTS

We divided all the reviewed articles into four categories, self-management (medication adherence, adherence to medical regimen, and remote monitoring), evaluation, interaction, and interface; 37.5% of the studies were focused on lung transplantation. In 56.25% of the studies, medication adherence was considered because one of the main reasons for the rejection and graft loss is stated medication nonadherence. Also, 62.5% of the studies demonstrated that the use of m-health improved medication adherence and self-management in transplantation.

CONCLUSIONS

The use of m-Health apps interventions to self-management after transplantation has shown promising feasibility and acceptability, and there is modest evidence to support the efficacy of these interventions. We found that m-Health solutions can help the patient in self-management in many ways after transplantation.

Usability Testing of a Digital Assessment Routing Tool: Protocol for an Iterative Convergent Mixed Methods Study

BACKGROUND

Musculoskeletal conditions account for 16% of global disability, resulting in a negative effect on millions of patients and an increasing burden on health care utilization. Digital technologies that improve health care outcomes and efficiency are considered a priority; however, innovations are often inadequately developed and poorly adopted. Further, they are rarely tested with sufficient rigor in clinical trials-the gold standard for clinical proof of efficacy. We have developed a new musculoskeletal Digital Assessment Routing Tool (DART) that allows users to self-assess and be directed to the right care. DART requires usability testing in preparation for clinical trials.

OBJECTIVE

This study will use the iterative convergent mixed methods design to assess and mitigate all serious usability issues to optimize user experience and adoption. Using this methodology, we will provide justifiable confidence to progress to full-scale randomized controlled trials when DART is integrated into clinical management pathways. This study protocol will provide a blueprint for future usability studies of mobile health solutions.

METHODS

We will collect qualitative and quantitative data from 20-30 participants aged 18 years and older for 4 months. The exact number of participants recruited will be dependent on the number of iterative cycles required to reach the study end points. Building on previous internal testing and stakeholder involvement, quantitative data collection is defined by the constructs within the ISO 9241-210-2019 standard and the system usability scale, providing a usability score for DART. Guided by the participant responses to quantitative questioning, the researcher will focus the qualitative data collection on specific usability problems. These will then be graded to provide the rationale for further DART system improvements throughout the iterative cycles.

RESULTS

This study received approval from the Queen Mary University of London Ethics of Research Committee (QMREC2018/48/048) on June 4, 2020. At manuscript submission, study recruitment was on-going, with data collection to be completed and results published in 2021.

CONCLUSIONS

This study will provide evidence concerning mobile health DART system usability and acceptance determining system improvements required to support user adoption and minimize suboptimal system usability as a potential confounder within subsequent noninferiority clinical trials. Success should produce a safe effective system with excellent usability, facilitating quicker and easier patient access to appropriate care while reducing the burden on primary and secondary care musculoskeletal services. This deliberately rigorous approach to mobile health innovation could be used as a guide for other developers of similar apps.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/27205.

Usability of Rapid Cholera Detection Device (OmniVis) for Water Quality Workers in Bangladesh: Iterative Convergent Mixed Methods Study

BACKGROUND

Cholera poses a significant global health burden. In Bangladesh, cholera is endemic and causes more than 100,000 cases each year. Established environmental reservoirs leave millions at risk of infection through the consumption of contaminated water. The Global Task Force for Cholera Control has called for increased environmental surveillance to detect contaminated water sources prior to human infection in an effort to reduce cases and deaths. The OmniVis rapid cholera detection device uses loop-mediated isothermal amplification and particle diffusometry detection methods integrated into a handheld hardware device that attaches to an iPhone 6 to identify and map contaminated water sources.

OBJECTIVE

The aim of this study was to evaluate the usability of the OmniVis device with targeted end users to advance the iterative prototyping process and ultimately design a device that easily integrates into users' workflow.

METHODS

Water quality workers were trained to use the device and subsequently completed an independent device trial and usability questionnaire. Pretraining and posttraining knowledge assessments were administered to ensure training quality did not confound trial and questionnaire.

RESULTS

Device trials identified common user errors and device malfunctions including incorrect test kit insertion and device powering issues. We did not observe meaningful differences in user errors or device malfunctions accumulated per participant across demographic groups. Over 25 trials, the mean time to complete a test was 47 minutes, a significant reduction compared with laboratory protocols, which take approximately 3 days. Overall, participants found the device easy to use and expressed confidence and comfort in using the device independently.

CONCLUSIONS

These results are used to advance the iterative prototyping process of the OmniVis rapid cholera detection device so it can achieve user uptake, workflow integration, and scale to ultimately impact cholera control and elimination strategies. We hope this methodology will promote robust usability evaluations of rapid pathogen detection technologies in device development.

A hand exercise mobile app for people with rheumatoid arthritis in Turkey: design, development and usability study

To design and develop a smartphone application for a structured hand exercise programme for patients with rheumatoid arthritis (RA) in Turkey and to test its usability. We followed a two-stage process: (1) Design and Development and (2) Usability testing. In stage 1, we used a qualitative user-centered design approach. We conducted a focus group (8 therapists and people with RA) to discuss the content, features and design to produce a prototype of the application. In a second focus group session, the participants tested the prototype, provided feedback and further revisions were made. In stage 2, 17 participants with RA used the app for 4 to 6 weeks. The System Usability Scale and the adapted Usability, Satisfaction and Ease to Use Questionnaires were used to measure usability, ease of use. Semi-structured interviews were conducted to explore user experiences with the application with 17 participants. In stage 1, the following themes were identified from the focus groups (a) login techniques (b) self-monitoring (c) exercises, (d) exercise diary, (e) information, (f) behavioral change and encouragement (g) exercise adherence. In stage 2, 3 themes were determined from interviews: (a) learning and accuracy, (b) ease of use, (c) motivation and adherence. USE and SUS scores indicated that users reported a high level of usability, satisfaction and ease of use. A mobile app for hand exercise for people with RA was developed using a mixed-method and iterative design. Participants perceived the mobile app as easy to use with high levels of satisfaction.

Usability of eHealth and Mobile Health Interventions by Young People Living With Juvenile Idiopathic Arthritis: Systematic Review

BACKGROUND

Considering the changing landscape of internet use and rising ownership of digital technology by young people, new methods could be considered to improve the current model of juvenile idiopathic arthritis (JIA) management.

OBJECTIVE

This systematic review aims to evaluate the usability of eHealth and mobile health (mHealth) interventions currently available for young people living with JIA.

METHODS

The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines were used to oversee this review. We systematically searched 15 databases for 252 potential studies; 2 authors independently screened all quantitative studies reporting the use of eHealth and mHealth interventions for young people (aged 1-18 years) diagnosed with JIA. Studies were excluded if they did not report outcome measures or were reviews, commentaries, or qualitative studies. Study methodological quality was scored using the Down and Black (modified) checklist. A narrative descriptive methodology was used to quantify the data because of heterogeneity across the studies.

RESULTS

A total of 11 studies were included in this review, reporting 7 eHealth and mHealth interventions for young people (aged 4-18 years) living with JIA, targeting health issues such as pain, health-related quality of life, physical activity, and chronic disease self-management. The usability of the interventions was facilitated through training and ongoing support. The engagement was promoted by a combination of persuasive influences, and barriers preventing adherence were removed through personal reminders and flexible program schedules to cater to JIA and non-JIA illnesses or other commonly seen activities in childhood. The feedback obtained was that most young people and their parents liked the interventions.

CONCLUSIONS

The results of this review need to be considered cautiously because of the lack of rigorous testing and heterogeneity, which limits the detailed descriptions of data synthesis. Further research is needed to consider gender differences, associated costs, and the effectiveness of the interventions on health outcomes to better support young people living with JIA.

Clinicians' perspectives on inertial measurement units in clinical practice

Inertial measurement units (IMUs) have been increasingly popular in rehabilitation research. However, despite their accessibility and potential advantages, their uptake and acceptance by health professionals remain a big challenge. The development of an IMU-based clinical tool must bring together engineers, researchers and clinicians. This study is part of a developmental process with the investigation of clinicians’ perspectives about IMUs. Clinicians from four rehabilitation centers were invited to a 30-minute presentation on IMUs. Then, two one-hour focus groups were conducted with volunteer clinicians in each rehabilitation center on: 1) IMUs and their clinical usefulness, and 2) IMUs data analysis and visualization interface. Fifteen clinicians took part in the first focus groups. They expressed their thoughts on: 1) categories of variables that would be useful to measure with IMUs in clinical practice, and 2) desired characteristics of the IMUs. Twenty-three clinicians participated to the second focus groups, discussing: 1) functionalities, 2) display options, 3) clinical data reported and associated information, and 4) data collection duration. Potential influence of IMUs on clinical practice and added value were discussed in both focus groups. Clinicians expressed positive opinions about the use of IMUs, but their expectations were high before considering using IMUs in their practice.

Features of a mobile health intervention to manage chronic obstructive pulmonary disease: a qualitative study

Background:

The use of mobile health (mHealth) interventions has the potential to enhance chronic obstructive pulmonary disease (COPD) treatment outcomes. Further research is needed to determine which mHealth features are required to potentially enhance COPD self-management.

Aim:

The aim of this study was to explore the potential features of an mHealth intervention for COPD management with healthcare providers (HCPs) and patients with COPD. It could inform the development and successful implementation of mHealth interventions for COPD management.

Methods:

This was a qualitative study. We conducted semi-structured individual interviews with HCPs, including nurses, pharmacists and physicians who work directly with patients with COPD. Interviews were also conducted with a diverse sample of patients with COPD. Interview topics included demographics, mHealth usage, the potential use of medical devices and recommendations for features that would enhance an mHealth intervention for COPD management.

Results:

A total of 40 people, including nurses, physicians and pharmacists, participated. The main recommendations for the proposed mHealth intervention were categorised into two categories: patient interface and HCP interface. The prevalent features suggested for the patient interface include educating patients, collecting baseline data, collecting subjective data, collecting objective data via compatible medical devices, providing a digital action plan, allowing patients to track their progress, enabling family members to access the mHealth intervention, tailoring the features based on the patient’s unique needs, reminding patients about critical management tasks and rewarding patients for their positive behaviours. The most common features of the HCP interface include allowing HCPs to track their patients’ progress, allowing HCPs to communicate with their patients, educating HCPs and rewarding HCPs.

Conclusion:

This study identifies important potential features so that the most effective, efficient and feasible mHealth intervention can be developed to improve the management of COPD.

The reviews of this paper are available via the supplemental material section.

Perceptions of Patients Regarding Mobile Health Interventions for the Management of Chronic Obstructive Pulmonary Disease: Mixed Methods Study

Background

Using a mobile health (mHealth) intervention consisting of a smartphone and compatible medical device has the potential to enhance chronic obstructive pulmonary disease (COPD) treatment outcomes while mitigating health care costs.

Objective

This study aims to describe the demographics, use, and access to smartphones of patients with COPD. It also aims to explore and develop an understanding of potential facilitators and barriers that might influence patients using mHealth interventions for COPD management.

Methods

This was an explanatory, sequential mixed methods study. Patients who attended respirology clinics completed a questionnaire on technology access and use. We conducted semistructured individual interviews with the patients. Interview topics included the following: demographics, mHealth use, perceptions toward challenges of mHealth adoption, factors facilitating mHealth adoption, and preferences regarding features of mHealth interventions for COPD management.

Results

A total of 100 adults completed the survey but 22 participants were excluded because they were not diagnosed with COPD. Of these, 10 patients with COPD participated in the interview. The quantitative component revealed that many patients with COPD owned a mobile phone, but only about one-fourth of the participants (18/77, 23%) owned a smartphone. The likelihood of owning a smartphone was not associated with age, sex, marital status, or geographical location, but patients with high educational status were more likely to own a smartphone. The qualitative component found that patients with COPD, in general, had a positive attitude toward mHealth adoption for COPD management, but several facilitators and barriers were identified. The main facilitators of mHealth adoption are possible health benefits for patients, ease of use, educating patients, and credibility. Alternatively, the barriers to adoption are technical issues, lack of awareness, potential limited uptake from older adults, privacy and confidentiality issues, finances, and lack of interest in mHealth

Conclusions

It is important to understand the perceptions of patients with COPD regarding the adoption of innovative mHealth interventions for COPD management. This study identifies some potential facilitators and barriers that may inform the successful development and implementation of mHealth interventions for COPD management.

Recommendations for Developing Support Tools With People Suffering From Chronic Obstructive Pulmonary Disease: Co-Design and Pilot Testing of a Mobile Health Prototype

BACKGROUND

Gaps exist between developers, commissioners, and end users in terms of the perceived desirability of different features and functionalities of mobile apps.

OBJECTIVE

The objective of this study was to co-design a prototype mobile app for people with chronic obstructive pulmonary disease (COPD). We present lessons learned and recommendations from working on a large project with various stakeholders to develop a mobile app for patients with COPD.

METHODS

We adopted a user-centered, participatory approach to app development. Following a series of focus groups and interviews to capture requirements, we developed a prototype app designed to enable daily symptom recording (experience sampling). The prototype was tested in a usability study applying the think aloud protocol with people with COPD. It was then released via the Android app store, and experience sampling data and event data were captured to gather further usability data.

RESULTS

A total of 5 people with COPD participated in the pilot study. Identified themes include familiarity with technology, appropriate levels for feeding back information, and usability issues such as manual dexterity. Moreover, 37 participants used the app over a 4-month period (median age 47 years). The symptoms most correlated to perceived well-being were tiredness (r=0.61; P<.001) and breathlessness (r=0.59; P<.001).

CONCLUSIONS

Design implications for COPD apps include the need for clearly labeled features (rather than relying on colors or symbols that require experience using smartphones), providing weather information, and using the same terminology as health care professionals (rather than simply lay terms). Target users, researchers, and developers should be involved at every stage of app development, using an iterative approach to build a prototype app, which should then be tested in controlled settings as well as in the wild (ie, when deployed and used in real-world settings) over longer periods.

Optimizing Child Nutrition Education With the Foodbot Factory Mobile Health App: Formative Evaluation and Analysis

BACKGROUND

Early nutrition interventions to improve food knowledge and skills are critical in enhancing the diet quality of children and reducing the lifelong risk of chronic disease. Despite the rise of mobile health (mHealth) apps and their known effectiveness for improving health behaviors, few evidence-based apps exist to help engage children in learning about nutrition and healthy eating.

OBJECTIVE

This study aimed to describe the iterative development and user testing of Foodbot Factory, a novel nutrition education gamified app for children to use at home or in the classroom and to present data from user testing experiments conducted to evaluate the app.

METHODS

An interdisciplinary team of experts in nutrition, education (pedagogy), and game design led to the creation of Foodbot Factory. First, a literature review and an environmental scan of the app marketplace were conducted, and stakeholders were consulted to define the key objectives and content of Foodbot Factory. Dietitian and teacher stakeholders identified priority age groups and learning objectives. Using a quasi-experimental mixed method design guided by the Iterative Convergent Design for Mobile Health Usability Testing approach, five app user testing sessions were conducted among students (ages 9-12 years). During gameplay, engagement and usability were assessed via direct observations with a semistructured form. After gameplay, qualitative interviews and questionnaires were used to assess user satisfaction, engagement, usability, and knowledge gained.

RESULTS

The environmental scan data revealed that few evidence-based nutrition education apps existed for children. A literature search identified key nutrients of concern for Canadian children and techniques that could be incorporated into the app to engage users in learning. Foodbot Factory included characters (2 scientists and Foodbots) who initiate fun and engaging dialogue and challenges (minigames), with storylines incorporating healthy eating messages that align with the established learning objectives. A total of five modules were developed: drinks, vegetables and fruit, grain foods, animal protein foods, and plant protein foods. Seven behavior change techniques and three unique gamified components were integrated into the app. Data from each user testing session were used to inform and optimize the next app iteration. The final user testing session demonstrated that participants agreed that they wanted to play Foodbot Factory again (12/17, 71%), that the app is easy to use (12/17, 71%) and fun (14/17, 88%), and that the app goals were clearly presented (15/17, 94%).

CONCLUSIONS

Foodbot Factory is an engaging and educational mHealth intervention for the Canadian public that is grounded in evidence and developed by an interdisciplinary team of experts. The use of an iterative development approach is a demonstrated method to improve engagement, satisfaction, and usability with each iteration. Children find Foodbot Factory to be fun and easy to use, and can engage children in learning about nutrition.

A Mobile Sexual Health App on Empowerment, Education, and Prevention for Young Adult Men (MyPEEPS Mobile): Acceptability and Usability Evaluation

BACKGROUND

HIV incidence among young adult men who have sex with men (MSM), particularly among black and Latino men, continues to rise. As such, continued HIV prevention interventions for young MSM of color are of utmost importance. Male Youth Pursuing Empowerment, Education and Prevention around Sexuality (MyPEEPS) Mobile is a comprehensive HIV prevention and sexual health education smartphone app initially created to promote sexual health and HIV prevention among adolescent sexual minority young men aged 13 to 18 years.

OBJECTIVE

The objective of this study was to critically appraise the acceptability and usability of MyPEEPS Mobile for young adult MSM aged 19 to 25 years.

METHODS

Study participants used the mobile app, completed usability questionnaires and in-depth interviews, and reported their experience using the app. Analysis of interview data was guided by the Unified Theory of Acceptance and Use of Technology (UTAUT) to better understand the usability and acceptability of this intervention for young adults. Interview data were coded using the following constructs from the UTAUT model: performance expectancy, effort expectancy, and social influence.

RESULTS

A total of 20 young adult MSM (n=10 in Chicago, Illinois, and n=10 in New York, New York) were enrolled in the study. Participants reported that MyPEEPS Mobile was free of functional problems (Health Information Technology Usability Evaluation Scale scores and Post-Study System Usability Questionnaire scores consistent with high usability), easy to use, and useful, with an engaging approach that increased acceptability, including the use of avatars and animation, and inclusive representation of the diverse identities by race and ethnicity, gender identity, and sexual orientation. Recommended areas for improving MyPEEPS Mobile for the target demographic included more adult-oriented graphics, advanced educational content, scenarios for youth with more sexual experience, and search function to increase accessibility of key content.

CONCLUSIONS

Overall, young adult MSM aged 19 to 25 years described the MyPEEPS Mobile as educational, informative, and usable for their sexual health education and HIV prevention needs, and they provided actionable recommendations to optimize its use and applicability for this age group.

Perceptions of Health Care Providers Regarding a Mobile Health Intervention to Manage Chronic Obstructive Pulmonary Disease: Qualitative Study

Background

Using a mobile health (mHealth) intervention, consisting of a smartphone and compatible medical device, has the potential to enhance chronic obstructive pulmonary disease (COPD) treatment outcomes while mitigating health care costs.

Objective

The aim of this study was to explore the potential facilitators and barriers among health care providers (HCPs) regarding the use of mHealth interventions for COPD management.

Methods

This was a qualitative study. Semistructured individual interviews were conducted with HCPs, including nurses, pharmacists, and physicians who work directly with patients with COPD. A flexible prompts guide was used to facilitate discussions. Interview topics included the following: demographics, mHealth usage, perceptions toward challenges of mHealth adoption, factors facilitating mHealth adoption, and preferences regarding features of the mHealth intervention for COPD management. Interviews were conversational in nature, and items were not asked verbatim or in the order presented. The interviews were transcribed verbatim and compared against the digital recordings to ensure the accuracy of the content. After creating a codebook for analysis, 2 researchers independently coded the remaining interview data using pattern coding. They discussed commonalities and differences in coding until a consensus was reached.

Results

A total of 30 nurses, physicians, and pharmacists participated. The main facilitators to mHealth adoption are possible health benefits for patients, ease of use, educating patients and their HCPs, credibility, and reducing cost to the health care system. Alternatively, the barriers to adoption are technical issues, privacy and confidentiality issues, lack of awareness, potential limited uptake from the elderly, potential limited connection between patients and HCPs, and finances.

Conclusions

It is important to understand the perceptions of HCPs regarding the adoption of innovative mHealth interventions for COPD management. This study identifies some potential facilitators and barriers that may inform the successful development and implementation of mHealth interventions for COPD management.