Usability Testing of the BRANCH Smartphone App Designed to Reduce Harmful Drinking in Young Adults

Evaluation of usability and user feedback to guide telepharmacy application development in Indonesia: a mixed-methods study

BACKGROUND

In Indonesia, the adoption of telepharmacy was propelled by the COVID-19 pandemic, prompting the need for a user-friendly application to support both the general population and pharmacists in accessing healthcare services. Therefore, this study aimed to evaluate usability and user feedback of a pioneering telepharmacy application known as Tanya Obat (translating to "Ask about Medications") in Indonesia, from the perspectives of the general population and pharmacists.

METHODS

A mixed-methods sequential study was conducted with the early-stage Tanya Obat application in Bandung City. Participants, including the general population and pharmacists, were instructed to use the application for a week. Questionnaires for the general population and pharmacists were distributed from March to May and February to June 2023, respectively. The System Usability Scale questionnaire was adopted to describe usability of the developed application. Further exploration of the quantitative results required collecting open-ended feedback to assess the impressions of the participants, difficulties encountered, and desired features for enhanced user-friendliness. The collected statements were summarized and clustered using thematic analysis. Subsequently, the association between the characteristics of participants and perceived usability was determined with the Chi-square test.

RESULT

A total of 176 participants, comprising 100 individuals from the general population and 76 pharmacists, engaged in this study. In terms of usability, the questionnaire showed that Tanya Obat application was on the borderline of acceptability, with mean scores of 63.4 and 64.1 from the general population and pharmacists, respectively. Additionally, open-ended feedback targeted at achieving a more compelling user experience was categorized into two themes, including concerns regarding the functionality of certain features and recommendations for improved visual aesthetics and bug fixes. No significant associations were observed between the characteristics of participants and perceived usability (p-value > 0.05).

CONCLUSION

The results showed that the perceived usability of Tanya Obat developed for telepharmacy was below average. Therefore, feature optimizations should be performed to facilitate usability of this application in Indonesia.

Design and usability evaluation of a mobile application for self-care among Iranian adolescents

BACKGROUND

Mobile phones can be an ideal platform to engage adolescents to maintain, improve, and promote self-care. Therefore, the current study aims to design and evaluate the usability of a mobile application for self-care in adolescents with a user-centered approach.

METHODS

The current applied developmental study was done in four steps. The first step, polling and examining opinions was conducted through in-depth semi-structured interviews, with the aim of user-centered mobile application design with the involvement of 30 participants. The second step, extracting and compiling the educational content related to the main themes of the self-care app, was obtained from national and international guidelines and instructions, including the World Health Organization, the Center for Disease Control and Prevention, the Ministry of Health and Medical Education, etc. In the third step, the initial version of the mobile application was developed. In the fourth step, app usability was evaluated by 30 participants from the target group, 2 weeks after using the app, using the MAUQ questionnaire.

RESULTS

In the first step, 789 codes, 12 sub-categories, and 3 categories were extracted. These codes were used in the design of the mobile application. In the second step, educational information was prepared and arranged in 5 sections (physical activity, nutrition, personal hygiene, risky behaviors and safety and events) in the form of text, images and short videos. In the third step, the mobile application was designed based on step 1 and 2. This application operates in online mode and under the Android operating system. the initial version of the mobile application was developed using JavaScript and Typescript programming languages in a Visual Studio Code environment. In the fourth step, the participants the overall level of usability of the application as very good with an average of 6.28 ± 0.55. The highest average score was given to the user interface and satisfaction with an average score of 6.43 ± 0.58.

CONCLUSIONS

The "My-Care" app is a collaboratively designed smartphone app for adolescents that targets 5 dimensions of physical self-care. This app has the potential to teach, assess, and promote self-care among adolescents.

U.S. veterans' experiences and factors associated with use of a smartphone application to self-manage unhealthy alcohol use

Unhealthy alcohol use is common among Operations Enduring and Iraqi Freedom (OEF/OIF) veterans, yet barriers discourage treatment-seeking. Mobile applications (apps) that deliver alcohol interventions have potential to address these barriers and increase treatment receipt. Few studies have qualitatively assessed users' experiences with apps to manage alcohol use. We assessed OEF/OIF veterans' experiences with Step Away, an app to reduce alcohol-related risks, to identify factors that may influence engagement. This single-arm pilot study recruited OEF/OIF veterans with positive alcohol screens nationwide using mail/telephone. Veterans aged 18-55 who exceeded drinking guidelines and owned an iPhone were eligible. Twenty-one (16 men, 5 women) of 55 participants completed interviews. Interviews were analyzed using thematic analysis. Participants found Step Away easy to use, although setup was time consuming. Participants reported increased awareness of alcohol use, highlighting daily assessment, weekly feedback, goal setting, and high-risk notification features as helpful and associated awareness with an intent to decrease use. Participants described Step Away as informative, with over half reporting they would use it outside of the study and most recommending it. Suggestions for improvement included greater personalization and control over features. Step Away features appear to influence engagement and increase users' awareness about alcohol consumed and factors associated with drinking, as well as intent to change. Assessment, feedback, and customization features of apps may facilitate app engagement. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Designing personalised mHealth solutions: An overview

INTRODUCTION

Mobile health, or mHealth, is based on mobile information and communication technologies and provides solutions for empowering individuals to participate in healthcare. Personalisation techniques have been used to increase user engagement and adherence to interventions delivered as mHealth solutions. This study aims to explore the current state of personalisation in mHealth, including its current trends and implementation.

MATERIALS AND METHODS

We conducted a review following PRISMA guidelines. Four databases (PubMed, ACM Digital Library, IEEE Xplore, and APA PsycInfo) were searched for studies on mHealth solutions that integrate personalisation. The retrieved papers were assessed for eligibility and useful information regarding integrated personalisation techniques.

RESULTS

Out of the 1,139 retrieved studies, 62 were included in the narrative synthesis. Research interest in the personalisation of mHealth solutions has increased since 2020. mHealth solutions were mainly applied to endocrine, nutritional, and metabolic diseases; mental, behavioural, or neurodevelopmental diseases; or the promotion of healthy lifestyle behaviours. Its main purposes are to support disease self-management and promote healthy lifestyle behaviours. Mobile applications are the most prevalent technological solution. Although several design models, such as user-centred and patient-centred designs, were used, no specific frameworks or models for personalisation were followed. These solutions rely on behaviour change theories, use gamification or motivational messages, and personalise the content rather than functionality. A broad range of data is used for personalisation purposes. There is a lack of studies assessing the efficacy of these solutions; therefore, further evidence is needed.

DISCUSSION

Personalisation in mHealth has not been well researched. Although several techniques have been integrated, the effects of using a combination of personalisation techniques remain unclear. Although personalisation is considered a persuasive strategy, many mHealth solutions do not employ it.

CONCLUSIONS

Open research questions concern guidelines for successful personalisation techniques in mHealth, design frameworks, and comprehensive studies on the effects and interactions among multiple personalisation techniques.

Development of "LvL UP 1.0": a smartphone-based, conversational agent-delivered holistic lifestyle intervention for the prevention of non-communicable diseases and common mental disorders

Background

Non-communicable diseases (NCDs) and common mental disorders (CMDs) are the leading causes of death and disability worldwide. Lifestyle interventions via mobile apps and conversational agents present themselves as low-cost, scalable solutions to prevent these conditions. This paper describes the rationale for, and development of, "LvL UP 1.0″, a smartphone-based lifestyle intervention aimed at preventing NCDs and CMDs.

Materials and Methods

A multidisciplinary team led the intervention design process of LvL UP 1.0, involving four phases: (i) preliminary research (stakeholder consultations, systematic market reviews), (ii) selecting intervention components and developing the conceptual model, (iii) whiteboarding and prototype design, and (iv) testing and refinement. The Multiphase Optimization Strategy and the UK Medical Research Council framework for developing and evaluating complex interventions were used to guide the intervention development.

Results

Preliminary research highlighted the importance of targeting holistic wellbeing (i.e., both physical and mental health). Accordingly, the first version of LvL UP features a scalable, smartphone-based, and conversational agent-delivered holistic lifestyle intervention built around three pillars: Move More (physical activity), Eat Well (nutrition and healthy eating), and Stress Less (emotional regulation and wellbeing). Intervention components include health literacy and psychoeducational coaching sessions, daily "Life Hacks" (healthy activity suggestions), breathing exercises, and journaling. In addition to the intervention components, formative research also stressed the need to introduce engagement-specific components to maximise uptake and long-term use. LvL UP includes a motivational interviewing and storytelling approach to deliver the coaching sessions, as well as progress feedback and gamification. Offline materials are also offered to allow users access to essential intervention content without needing a mobile device.

Conclusions

The development process of LvL UP 1.0 led to an evidence-based and user-informed smartphone-based intervention aimed at preventing NCDs and CMDs. LvL UP is designed to be a scalable, engaging, prevention-oriented, holistic intervention for adults at risk of NCDs and CMDs. A feasibility study, and subsequent optimisation and randomised-controlled trials are planned to further refine the intervention and establish effectiveness. The development process described here may prove helpful to other intervention developers.

Development of a Secondary Prevention Smartphone App for Students With Unhealthy Alcohol Use: Results From a Qualitative Assessment

BACKGROUND

Despite considerable efforts devoted to the development of prevention interventions aiming at reducing unhealthy alcohol use in tertiary students, their delivery remains often challenging. Interventions including information technology are promising given their potential to reach large parts of the population.

OBJECTIVE

This study aims to develop a secondary prevention smartphone app with an iterative qualitative design involving the target population.

METHODS

The app development process included testing a first prototype and a second prototype, developed based on the results of 2 consecutive qualitative assessments. Participants (aged ≥18 years, screened positive for unhealthy alcohol use) were students from 4 tertiary education institutions in the French-speaking part of Switzerland. Participants tested prototype 1 or prototype 2 or both and provided feedback in 1-to-1 semistructured interviews after 2-3 weeks of testing.

RESULTS

The mean age of the participants was 23.3 years. A total of 9 students (4/9 female) tested prototype 1 and participated in qualitative interviews. A total of 11 students (6/11 female) tested prototype 2 (6 who tested prototype 1 and 5 new) and participated in semistructured interviews. Content analysis identified 6 main themes: "General Acceptance of the App," "Importance of the Targeted and Relevant App Content," "Importance of Credibility," "Importance of the App Usability," "Importance of a Simple and Attractive Design," "Importance of Notifications to Ensure App Use over Time." Besides a general acceptance of the app, these themes reflected participants' recommendations toward increased usability; to improve the design; to include useful and rewarding contents; to make the app look serious and credible; and to add notifications to ensure its use over time. A total of 11 students tested prototype 2 (6 who tested prototype 1 and 5 new) and participated in semistructured interviews. The 6 same themes emerged from the analysis. Participants from phase 1 generally found the design and content of the app improved.

CONCLUSIONS

Students recommend prevention smartphone apps to be easy to use, useful, rewarding, serious, and credible. These findings may be important to consider when developing prevention smartphone apps to increase the likelihood of app use over time.

TRIAL REGISTRATION

ISRCTN registry 10007691; https://www.isrctn.com/ISRCTN10007691.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.1186/s13063-020-4145-2.

Factors associated with young adult engagement with a web-based sickle cell reproductive health intervention

Objective

To determine the factors predicting the engagement of young adults who have sickle cell disease (SCD) or sickle cell trait (SCT) with an online reproductive health education intervention and engagement effects on knowledge.

Methods

The cross-sectional study included 167 participants who completed the web-based intervention either face-to-face (F2F) or online delivery (OL). Measures include: time used relative to length of the intervention narration and media (engagement) and the SCKnowIQ questionnaire. Ordinal regression was conducted.

Results

The sample mean age was 26-years (SD=5), 68% were female, 54% had SCD, and 68% were in the F2F group. Adjusting for age, partner sickle cell status, marital status, and education, participants who were female (p=.003), had SCD (p=.018), or had F2F delivery (p < .001) were more likely to spend more time on the intervention. Adjusting for baseline knowledge and modality, more time spent on the intervention was associated with higher posttest knowledge (p=.006).

Conclusions

Future studies are necessary to understand reasons underpinning engagement and to investigate other unmeasured factors, such as intervention interactivity elements, that could also be associated with engagement.

Innovation

This study of young adults with SCD or SCT provides much needed insight about their engagement with online reproductive health education.

Think-Aloud Testing of a Novel Safer Drinking App for College Students During COVID-19: Usability Study

BACKGROUND

Hazardous alcohol consumption, and binge drinking in particular, continues to be common among college students, posing the greatest risk for their health and safety. Despite widespread exposure to evidence-based preventive interventions among US undergraduates, only modest and temporary effects on risky drinking occur. Formative studies have demonstrated that students want a more engaging intervention tool for risky drinking that can be used just in time.

OBJECTIVE

The purpose of this study is to test the appeal, relevance, and perceived utility of a draft mobile app for safer student drinking at a public university in Virginia.

METHODS

Undergraduate student participants tested the draft mobile app via a web-based prototype that tailors to individual feedback with hot spots that responded to their taps to mimic app functionality. They narrated their impressions, navigation, and comments in a standardized think-aloud procedure. After each round of think-aloud interviews, researchers debriefed the investigators and developers to discuss findings and brainstorm app modifications.

RESULTS

Minor changes to the functionality and aesthetics would improve usability of the app (eg, option for light mode in app settings). Student testers recommended tailoring the app to the needs of college students and to aspects of the local university's drinking culture.

CONCLUSIONS

Findings from this study will be synthesized with information gained from other formative work to determine the final app features. We will test the app in a pilot randomized trial to assess app use and the impact of the app on college student drinking behavior over several months.

An Electronic Patient-Reported Outcome Mobile App for Data Collection in Type A Hemophilia: Design and Usability Study

Background

There is currently limited evidence on the level and intensity of physical activity in individuals with hemophilia A. Mobile technologies can offer a rigorous and reliable alternative to support data collection processes but they are often associated with poor user retention. The lack of longitudinal continuity in their use can be partly attributed to the insufficient consideration of stakeholder inputs in the development process of mobile apps. Several user-centered models have been proposed to guarantee that a thorough knowledge of the end user needs is considered in the development process of mobile apps.

Objective

The aim of this study is to design and validate an electronic patient-reported outcome mobile app that requires sustained active input by individuals during POWER, an observational study that aims at evaluating the relationship between physical activity levels and bleeding in patients with hemophilia A.

Methods

We adopted a user-centered design and engaged several stakeholders in the development and usability testing of this mobile app. During the concept generation and ideation phase, we organized a need-assessment focus group (FG) with patient representatives to elicit specific design requirements for the end users. We then conducted 2 exploratory FGs to seek additional inputs for the app’s improvement and 2 confirmatory FGs to validate the app and test its usability in the field through the mobile health app usability questionnaire.

Results

The findings from the thematic analysis of the need-assessment FG revealed that there was a demand for sense making, for simplification of app functionalities, for maximizing integration, and for minimizing the feeling of external control. Participants involved in the later stages of the design refinement contributed to improving the design further by upgrading the app’s layout and making the experience with the app more efficient through functions such as chatbots and visual feedback on the number of hours a wearable device had been worn, to ensure that the observed data were actually registered. The end users rated the app highly during the quantitative assessment, with an average mobile health app usability questionnaire score of 5.32 (SD 0.66; range 4.44-6.23) and 6.20 (SD 0.43; range 5.72-6.88) out of 7 in the 2 iterative usability testing cycles.

Conclusions

The results of the usability test indicated a high, growing satisfaction with the electronic patient-reported outcome app. The adoption of a thorough user-centered design process using several types of FGs helped maximize the likelihood of sustained retention of the app’s users and made it fit for data collection of relevant outcomes in the observational POWER study. The continuous use of the app and the actual level of engagement will be evaluated during the ongoing trial.

Trial Registration

ClinicalTrials.gov NCT04165135; https://clinicaltrials.gov/ct2/show/NCT04165135

Preconception and Diabetes Information (PADI) App for Women with Pregestational Diabetes: a Feasibility and Acceptability Study

Diabetes mellitus increases the risk of adverse maternal and fetal outcomes. Preconception care is vital to minimise complications; however, preconception care service provision is hindered by inadequate knowledge, resources and care fragmentation. Mobile health technology, particularly smartphone apps, could improve preconception care and pregnancy outcomes for women with diabetes. The aim of this study is to co-create a preconception and diabetes information app with healthcare professionals and women with diabetes and explore the feasibility, acceptability and preliminary effects of the app. A mixed-methods study design employing questionnaires and semi-structured interviews was used to assess preliminary outcome estimates (preconception care knowledge, attitudes and behaviours), and user acceptability. Data analysis included thematic analysis, descriptive statistics and non-parametric tests. Improvements were recorded in knowledge and attitudes to preconception care and patient activation measure following the 3-month app usage. Participants found the app acceptable (satisfaction rating was 72%), useful and informative. The app's usability and usefulness facilitated usage while manual data input and competing priorities were barriers which participants felt could be overcome via personalisation, automation and use of daily reminders. This is the first study to explore the acceptability and feasibility of a preconception and diabetes information app for women with diabetes. Triangulated data suggest that the app has potential to improve preconception care knowledge, attitudes and behaviours. However, in order for women with DM to realise the full potential of the app intervention, particularly improved maternal and fetal outcomes, further development and evaluation is required.

The Impact of Gamification-Induced Users' Feelings on the Continued Use of mHealth Apps: A Structural Equation Model With the Self-Determination Theory Approach

Background

Continued use of mHealth apps can achieve better effects in health management. Gamification is an important factor in promoting users’ intention to continue using mHealth apps. Past research has rarely explored the factors underlying the continued use of mobile health (mHealth) apps and gamification’s impact mechanism or path on continued use.

Objective

This study aimed to explore the factors influencing mHealth app users’ intention to continue using mHealth apps and the impact mechanism and path of users’ feelings induced by gamification on continued mHealth app use.

Methods

First, based on the expectation confirmation model of information system continuance, we built a theoretical model for continued use of mHealth apps based on users’ feelings toward gamification. We used self-determination theory to analyze gamification’s impact on user perceptions and set the resulting feelings (competence, autonomy, and relatedness) as constructs in the model. Second, we used the survey method to validate the research model, and we used partial least squares to analyze the data.

Results

A total of 2988 responses were collected from mHealth app users, and 307 responses were included in the structural equation model after passing the acceptance criteria. The intrinsic motivation for using mHealth apps is significantly affected by autonomy (β=.312; P<.001), competence (β=.346; P<.001), and relatedness (β=.165; P=.004) induced by gamification. The intrinsic motivation for using mHealth apps has a significant impact on satisfaction (β=.311, P<.001) and continuance intention (β=.142; P=.045); furthermore, satisfaction impacts continuance intention significantly (β=.415; P<.001). Confirmation has a significant impact on perceived usefulness (β=.859; P<.001) and satisfaction (β=.391; P<.001), and perceived usefulness has a significant impact on satisfaction (β=.269; P<.001) and continuance intention (β=.273; P=.001). The mediating effect analysis showed that in the impact path of the intrinsic motivation for using the mHealth apps on continuance intention, satisfaction plays a partial mediating role (β=.129; P<.001), with a variance accounted for of 0.466.

Conclusions

This study explored the impact path of users’ feelings induced by gamification on the intention of continued mHealth app use. We confirmed that perceived usefulness, confirmation, and satisfaction in the classical continued use theory for nonmedical information systems positively affect continuance intention. We also found that the path and mechanism of users' feelings regarding autonomy, competence, and relatedness generated during interactions with different gamification elements promote the continued use of mHealth apps.

The impact of celebrity influence and national media coverage on users of an alcohol reduction app: a natural experiment

BACKGROUND

Smartphone apps are increasingly used for health-related behaviour change and people discover apps through different sources. However, it is unclear whether users differ by mode of app discovery. Drink Less is an alcohol reduction app that received national media coverage in the UK caused by celebrity influence (a male TV and radio national broadcaster, aged 51). Our aim was to compare users who discovered the app before and after this coverage.

METHODS

A natural experiment assessing the impact of media coverage of Drink Less on users' socio-demographic and drinking characteristics, app engagement levels, and extent of alcohol reduction. The study period was from 17th May 2017 to 23rd January 2019, with media coverage starting on 21st August 2018. Users were 18 years or over, based in the UK and interested in drinking less. Interrupted time series analyses using Generalised Additive Mixed Models were conducted for each outcome variable aggregated at the weekly level.

RESULTS

In 66 weeks prior to the media coverage, 8617 users downloaded the app and 18,959 in 23 weeks afterwards. There was a significant step-level increase in users' mean age (B = 8.17, p < .001) and a decrease in the percentage of female users (B = -27.71, p < .001), though these effects dissipated non-linearly over time. No effect of media coverage was detected on employment type or on the percentage of at-risk drinkers, though the mean Alcohol Use Disorders Identification Test score was lower after the media coverage (B = -1.43, p = .031). There was a step-level increase in app engagement - number of sessions (B = 3.45, p = .038) and number of days used (B = 2.30, p = .005) - which continued to increase over time following quadratic trends.

CONCLUSIONS

Celebrity influence leading to national media coverage in the UK of the Drink Less app was associated with more people downloading the app who were male, older and engaged with the app; and did not appear to impact employment inequality.

Qualitative Evaluation of Web-Based Digital Intervention to Prevent and Reduce Excessive Alcohol Use and Harm Among Young People Aged 14-15 Years: A "Think-Aloud" Study

BACKGROUND

In the United Kingdom, despite some downward trends in alcohol use among young people, over one-fifth of young people reported excessive alcohol use in the past month, which is associated with short- and long-term harm to health. Digital interventions to reduce alcohol use, such as websites, among young people present an appealing and cost-effective mode of intervention that can be integrated into the education system. However, relatively few school-based digital alcohol-focused interventions have been developed and evaluated for young people in the United Kingdom.

OBJECTIVE

This study aims to develop a novel web-based intervention, Rethink Alcohol, to prevent and reduce excessive alcohol use and related harm among young people aged between 14 and 15 years, and explore the views of young people, teachers, and youth workers in relation to the content, design, and usability of the intervention.

METHODS

Intervention development followed the person-based approach, using theories of social norms and social influence. Qualitative "Think-Aloud" interviews, either one-to-one or paired, were conducted while participants perused and worked through the web-based intervention, talking aloud. Participants included 20 young people (12 female, 8 male), 5 youth workers (4 female, 1 male), 3 teachers (2 male, 1 female), and 1 (male) clinical professional, recruited via youth groups and professional networks. Interviews were audio recorded, transcribed verbatim, and analyzed thematically.

RESULTS

The prototype web-based intervention included normative feedback, information, a quiz, interactive activities, and scenarios. On a rating scale of impressions from poor (1) to excellent (5), participants gave an average score of 3.6/5. A total of 5 themes were identified: content, credibility of the website, making the website easy to understand, design and navigation, and suitability for the audience. These themes reflected views that the content was interesting, credible, informative, and embodied a neutral and nonjudgmental tone, but stronger messaging was needed regarding social pressures and short-term risks regarding safety and risk behavior alongside clarity around pathways of risk; credibility and trustworthiness of information were critical features, determined in part, by the professionalism of design and referencing of sources of information provided; and messages should be succinct and come to life through design and interactive features.

CONCLUSIONS

Together, the data illustrated the importance and challenge of communicating nuanced alcohol-focused public health messages to young people in concise, clear, nonjudgmental, and appealing ways. Young people report interest in clear, credible, neutral, and interactive messages regarding social pressures and short-term risks of alcohol use via a web-based intervention. There is scope for optimization and feasibility testing of the Rethink Alcohol intervention.

Design Features for Improving Mobile Health Intervention User Engagement: Systematic Review and Thematic Analysis

Background

Well-designed mobile health (mHealth) interventions support a positive user experience; however, a high rate of disengagement has been reported as a common concern regarding mHealth interventions. To address this issue, it is necessary to summarize the design features that improve user engagement based on research over the past 10 years, during which time the popularity of mHealth interventions has rapidly increased due to the use of smartphones.

Objective

The aim of this review was to answer the question “Which design features improve user engagement with mHealth interventions?” by summarizing published literature with the purpose of guiding the design of future mHealth interventions.

Methods

This review followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist. Databases, namely, PubMed, Web of Science, Cochrane Library, Ovid EMBASE, and Ovid PsycINFO, were searched for English and Chinese language papers published from January 2009 to June 2019. Thematic analysis was undertaken to assess the design features in eligible studies. The Mixed Methods Appraisal Tool was used to assess study quality.

Results

A total of 35 articles were included. The investigated mHealth interventions were mainly used in unhealthy lifestyle (n=17) and chronic disease (n=10) prevention programs. Mobile phone apps (n=24) were the most common delivery method. Qualitative (n=22) and mixed methods (n=9) designs were widely represented. We identified the following 7 themes that influenced user engagement: personalization (n=29), reinforcement (n=23), communication (n=20), navigation (n=17), credibility (n=16), message presentation (n=16), and interface aesthetics (n=7). A checklist was developed that contained these 7 design features and 29 corresponding specific implementations derived from the studies.

Conclusions

This systematic review and thematic synthesis identified useful design features that make an mHealth intervention more user friendly. We generated a checklist with evidence-based items to enable developers to use our findings easily. Future evaluations should use more robust quantitative approaches to elucidate the relationships between design features and user engagement.

LionVu 2.0 Usability Assessment for Pennsylvania, United States

The Penn State Cancer Initiative implemented LionVu 1.0 (Penn State University, United States) in 2017 as a web-based mapping tool to educate and inform public health professionals about the cancer burden in Pennsylvania and 28 counties in central Pennsylvania, locally known as the catchment area. The purpose of its improvement, LionVu 2.0, was to assist investigators answer person-place-time questions related to cancer and its risk factors by examining several data variables simultaneously. The primary objective of this study was to conduct a usability assessment of a prototype of LionVu 2.0 which included area- and point-based data. The assessment was conducted through an online survey; 10 individuals, most of whom had a masters or doctorate degree, completed the survey. Although most participants had a favorable view of LionVu 2.0, many had little to no experience with web mapping. Therefore, it was not surprising to learn that participants wanted short 10-15-minute training videos to be available with future releases, and a simplified user-interface that removes advanced functionality. One unexpected finding was the suggestion of using LionVu 2.0 for teaching and grant proposals. The usability study of the prototype of LionVu 2.0 provided important feedback for its future development.

Development of an Intervention Targeting Multiple Health Behaviors Among High School Students: Participatory Design Study Using Heuristic Evaluation and Usability Testing

Background

Mobile electronic platforms provide exciting possibilities for health behavior promotion. For instance, they can promote smoking cessation, moderate alcohol consumption, healthy eating, and physical activity. Young adults in Sweden are proficient in the use of technology, having been exposed to computers, smartphones, and the internet from an early age. However, with the high availability of mobile health (mHealth) interventions of varying quality, it is critical to optimize the usability of mHealth interventions to ensure long-term use of these health promotion interventions.

Objective

This study aims to investigate the usability of an mHealth intervention (LIFE4YOUth) targeting health behaviors among high school students through heuristic evaluation and usability testing.

Methods

A preliminary version of the LIFE4YOUth mHealth intervention, which was aimed at promoting healthy eating, physical activity, smoking cessation, and nonrisky drinking among high school students, was developed in early 2019. We completed a total of 15 heuristic evaluations and 5 usability tests to evaluate the usability of the mHealth intervention prototype to improve its functioning, content, and design.

Results

Heuristic evaluation from a total of 15 experts (10 employees and 5 university students, both women and men, aged 18-25 years) revealed that the major usability problems and the worst ratings, a total of 17 problems termed usability catastrophes, concerned shortcomings in displaying easy-to-understand information to the users or technical errors. The results of the usability testing including 5 high school students (both girls and boys, aged 15-18 years) showed that the design, quality, and quantity of content in the intervention may impact the users’ level of engagement. Poor functionality was considered a major barrier to usability. Of the 5 participants, one rated the LIFE4YOUth intervention as poor, 2 rated as average, and 2 assessed it as good, according to the System Usability Scale.

Conclusions

High school students have high expectations of digital products. If an mHealth intervention does not offer optimal functions, they may cease to use it. Optimizing the usability of mHealth interventions is a critical step in the development process. Heuristic evaluation and usability testing in this study provided valuable knowledge about the prototype from a user’s perspective. The findings may lead to the development of similar interventions targeting the high school population.

Proposal for the User-Centered Design Approach for Health Apps Based on Successful Experiences: Integrative Review

Background

Different strategies encompassed within mHealth have shown themselves to be effective for maintaining good health or controlling certain diseases. However, there is usually a very high rate of abandonment of health apps. Therefore, it would seem obvious that there is a need for involving the end users (whether they are health professionals, patients, or both) in the design process from the early stages in order to enable their needs and characteristics to be identified. In this sense, it is common knowledge that focusing on the user permits the consideration of valuable details aimed at making the correct adjustment between the patient, the technology, and the organization of attention.

Objective

The goal of the research was to propose a methodology based on the review of previous successful user experiences in setting up health apps by using qualitative techniques (focus groups and discussion groups) that includes the participation of information technology and health professionals and the patients themselves.

Methods

An integrative review was made of studies in which a qualitative methodology was employed mainly through focus and/or discussion groups for the design and development of health apps, consulting diverse databases (PubMed, Scopus, and Proquest) with the following search strategy: “mHealth AND apps AND focus group OR discussion group.” A total of 69 papers were included in the review.

Results

A proposal structured in 4 sessions of variable duration was made in which information technology and health professionals and patients take part: composing, preparing, and organizing contents (session 1); testing structure and usability (session 2); does the app fit the needs of end users? (session 3); and last testing—keep on improving (session 4). Throughout the sessions, we propose studying aspects like previous user experiences in mHealth, barriers to the adoption of mHealth, interface contents, management and browsability, usability, perceived quality, security and privacy, capacity to self-manage disease with the app, ergonomics, and glanceability, etc. Specific tools that have proved useful in previous research for measuring these aspects are presented.

Conclusions

These work sessions would be based on predominantly qualitative methodologies although, as they evolve, validated questionnaires permitting the assessment of the objectivity of certain technical aspects could be incorporated. With this proposal, a project centered on end users could be effected, responding to their needs. However, this requires validation that will be made via implementation in the development of health apps, with the subsequent measurement of results in terms of adherence and improvement in the clinical variables of the end users.

Iterative four-phase development of a theory-based digital behaviour change intervention to reduce occupational sedentary behaviour

Introduction

As high amounts of occupational sitting have been associated with negative health consequences, designing workplace interventions to reduce sedentary behaviour (SB) is of public health interest. Digital technology may serve as a cost-effective and scalable platform to deliver such an intervention. This study describes the iterative development of a theory-based, digital behaviour change intervention to reduce occupational SB.

Methods

The behaviour change wheel and The Behaviour Change Technique Taxonomy were used to guide the intervention design process and form a basis for selecting the intervention components. The development process consisted of four phases: phase 1 – preliminary research, phase 2 – consensus workshops, phase 3 – white boarding and phase 4 – usability testing.

Results

The process led to the development and refinement of a smartphone application – Worktivity. The core component was self-monitoring and feedback of SB at work, complemented by additional features focusing on goal setting, prompts and reminders to break up prolonged periods of sitting, and educational facts and tips. Key features of the app included simple data entry and personalisation based on each individual’s self-reported sitting time. Results from the ‘think-aloud’ interviews (n=5) suggest Worktivity was well accepted and that users were positive about its features.

Conclusion

This study led to the development of Worktivity, a theory-based and user-informed mobile app intervention to reduce occupational SB. It is the first app of its kind developed with the primary aim of reducing occupational SB using digital self-monitoring. This paper provides a template to guide others in the development and evaluation of technology-supported behaviour change interventions.

Usability of a Mobile Phone App Aimed at Adolescents and Young Adults During and After Cancer Treatment: Qualitative Study

Background

Adolescent and young adult (AYA) cancer patients are seldom involved in the process of testing cancer-related apps. As such, knowledge about youth-specific content, functionalities, and design is sparse. As a part of a co-creation process of developing the mobile phone app Kræftværket, AYAs in treatment for cancer and in follow-up participated in a usability think-aloud test of a prototype of the app. Thus, the app was initiated, created, and evaluated by AYAs with cancer experience.

Objective

The aim of this study was to explore the results of a think-aloud test administered to see how the prototype of the app Kræftværket was used by AYAs in treatment for cancer and in follow-up, and to investigate the strengths and weaknesses of the app.

Methods

A total of 20 AYA cancer patients aged 16 to 29 years (n=10 on treatment, n=10 in follow-up) were provided with the first version of the co-created mobile phone app Kræftværket during a 6-week test period (April-May 2018). After the test period, 15 participated in individual usability think-aloud tests. The tests were video-recorded, transcribed verbatim, and analyzed using a thematic analysis approach.

Results

The thematic analysis led to the following themes and subthemes: navigation (subthemes: intuition, features, buttons, home page, profile), visual and graphic design (subthemes: overview, text and colors, photos, videos, YouTube), and usefulness (subthemes: notifications, posts, adding). The analysis identified gender differences in app utilization—female participants seemed to be more familiar with parts of the app. The app seemed to be more relevant to AYAs receiving treatment due to app functions such as tracking symptoms and searching for relevant information. Lack of notifications and incorrect counting of posts were perceived as barriers to using the app.

Conclusions

Usability testing is crucial to meet the needs of the AYA target audience. AYA cancer apps should preferably be relevant, targeted, and unique, and include a tracking function and AYA-produced videos. Notifications and correct marking and ordering of posts are critical to make apps engaging and dynamic. Further research is recommended to evaluate the Kræftværket app with the input of more AYAs.

A Qualitative Evaluation of the Acceptability of a Tailored Smartphone Alcohol Intervention for a Military Population: Information About Drinking for Ex-Serving Personnel (InDEx) App

Background

Alcohol consumption in the UK Armed Forces is higher than in the general population, and this pattern continues after leaving the service. Smartphone apps may be useful to increase ex-serving personnel’s awareness of their alcohol consumption, support self-monitoring, and prompt a change in behavior.

Objective

The study aimed to explore the acceptability of Information about Drinking in Ex-serving personnel (InDEx), a tailored smartphone app, combined with personalized short message service (SMS) text messaging designed to target ex-serving personnel who meet the criteria for hazardous alcohol use.

Methods

The InDEx intervention included 4 key modules: (1) assessment and normative feedback, (2) self-monitoring and feedback, (3) goal setting and review, and (4) personalized SMS text messaging. A semistructured telephone interview study was conducted with ex-serving personnel after using the app for a 28-day period. Interviews were used to explore the acceptability of app modules and its functionality and the perceived changes in participant’s drinking. Interview transcripts were analyzed using inductive thematic analysis.

Results

Overall, 94% (29/31) participants who used InDEx agreed to take part in a telephone interview. Overall, 4 themes were identified: Credibility, Meeting their needs, Simplicity, and Helpful for ex-serving personnel. The importance of credibility, functionality, and meeting the individual needs of ex-serving personnel was emphasized. Acceptability and engagement with specific modules of the app and text messages were influenced by the following: (1) if they felt it provided credible information, (2) whether the content was appropriately personalized to them, (3) the ease of use, and (4) beliefs about their own drinking behaviors. Participants recommended that the app would be most suitable for personnel about to leave the Armed Forces.

Conclusions

InDEx was an acceptable smartphone app for ex-serving personnel for monitoring alcohol consumption and in providing meaningful feedback to the individual. Pages that met the participant’s interests and provided real time personalized, credible feedback on their drinking and text messages tailored to participant’s interactions with the app were particularly favored.

Measuring the Implementation of Behavioral Intervention Technologies: Recharacterization of Established Outcomes

Behavioral intervention technologies (BITs) are websites, software, mobile apps, and sensors designed to help users address or change behaviors, cognitions, and emotional states. BITs have the potential to transform health care delivery, and early research has produced promising findings of efficacy. BITs also favor new models of health care delivery and provide novel data sources for measurement. However, there are few examples of successful BIT implementation and a lack of consensus on as well as inadequate descriptions of BIT implementation measurement. The aim of this viewpoint paper is to provide an overview and characterization of implementation outcomes for the study of BIT use in routine practice settings. Eight outcomes for the evaluation of implementation have been previously described: acceptability, adoption, appropriateness, feasibility, fidelity, implementation cost, penetration, and sustainability. In a proposed recharacterization of these outcomes with respect to BIT implementation, definitions are clarified, expansions to the level of analysis are identified, and unique measurement characteristics are discussed. Differences between BIT development and implementation, an increased focus on consumer-level outcomes, the expansion of providers who support BIT use, and the blending of BITs with traditional health care services are specifically discussed. BITs have the potential to transform health care delivery. Realizing this potential, however, will hinge on high-quality research that consistently and accurately measures how well such technologies have been integrated into health services. This overview and characterization of implementation outcomes support BIT research by identifying and proposing solutions for key theoretical and practical measurement challenges.

Developing Typologies of User Engagement With the BRANCH Alcohol-Harm Reduction Smartphone App: Qualitative Study

BACKGROUND

Understanding how users engage with electronic screening and brief intervention (eSBI) is a critical research objective to improve effectiveness of app-based interventions to reduce harmful alcohol consumption. Although quantitative measures of engagement provide a strong indicator of how the user engages with an app at the group level, they do not elucidate finer-grained details of how apps function from an individual, experiential perspective and why, or how, users engage with an intervention in a particular manner.

OBJECTIVE

The aim of this study was to (1) understand why and how participants engaged with the BRANCH app, (2) explore facilitators and barriers to engagement with app features, (3) explore how the BRANCH app impacted drinking behavior, (4) use these data to identify typologies of users of the BRANCH app in terms of engagement behaviors, and (5) identify future eSBI app design implications.

METHODS

In total, 20 one-to-one semistructured telephone interviews were conducted with participants recruited from a randomized controlled trial, which evaluated the effectiveness of engagement-promoting strategies in the BRANCH app targeting harmful drinking in young adults (aged 18-30 years). The topic guide explored users' current engagement levels with existing health promotion apps, their views toward the effectiveness of such apps, and what they liked and disliked about BRANCH, specifically focusing on how they engaged with the app. Framework analysis was used to develop typologies of user engagement.

RESULTS

The study identified 3 typologies of engagers. Trackers were defined by their motivations to use health-tracking apps to monitor and understand quantified self-data. They did not have intentions necessarily to cut down and predominantly used only the drinking diary. Cut-downers were motivated to use the app because they wanted to reduce their alcohol consumption Unlike Trackers, they did not use a range of different health apps daily, but saw the BRANCH app as an opportunity to test out a different method of trying to cut down their alcohol use. This typology used more features than Trackers, such as the goal setting function. Noncommitters were characterized as a group of users who were initially enthusiastic about using the app; however, this enthusiasm quickly waned and they gained no benefit from it.

CONCLUSIONS

This was the first study to identify typologies of user engagement with eSBI apps. Although in need of replication, it provides a first step in understanding independent categories of eSBI users, who may benefit from apps tailored to a user's typology or motivation. It also provides new evidence to suggest that apps may be used more effectively as a tool to raise awareness of drinking, instead of reducing alcohol use, and be a step in the care pathway, identifying at-risk individuals and signposting them to more intensive treatment.

TRIAL REGISTRATION

International Standard Randomised Controlled Trial Number ISRCTN70980706; http://www.isrctn.com /ISRCTN70980706 (Archived by WebCite at http://www.webcitation.org/73vfDXYEZ).

A Smartphone App and Personalized Text Messaging Framework (InDEx) to Monitor and Reduce Alcohol Use in Ex-Serving Personnel: Development and Feasibility Study

BACKGROUND

Self-reported alcohol misuse remains high in armed forces personnel even after they have left service. More than 50% of ex-serving personnel meet the criteria for hazardous alcohol use; however, many fail to acknowledge that they have a problem. Previous research indicates that interventions delivered via smartphone apps are suitable in promoting self-monitoring of alcohol use, have a broad reach, and may be more cost-effective than other types of brief interventions. There is currently no such intervention specifically designed for the armed forces.

OBJECTIVE

This study sought to describe the development of a tailored smartphone app and personalized text messaging (short message service, SMS) framework and to test the usability and feasibility (measured and reported as user engagement) of this app in a hard-to-engage ex-serving population.

METHODS

App development used Agile methodology (an incremental, iterative approach used in software development) and was informed by behavior change theory, participant feedback, and focus groups. Participants were recruited between May 2017 and June 2017 from an existing United Kingdom longitudinal military health and well-being cohort study, prescreened for eligibility, and directed to download either Android or iOS versions of the "Information about Drinking for Ex-serving personnel" (InDEx) app. Through the app, participants were asked to record alcohol consumption, complete a range of self-report measures, and set goals using implementation intentions (if-then plans). Alongside the app, participants received daily automated personalized text messages (SMS) corresponding to specific behavior change techniques with content informed by the health action process approach with the intended purpose of promoting the use of the drinks diary, suggesting alternative behaviors, and providing feedback on goals setting.

RESULTS

Invitations to take part in the study were sent to ex-serving personnel, 22.6% (31/137) of whom accepted and downloaded the app. Participants opened the InDEx app a median of 15.0 (interquartile range [IQR] 8.5-19.0) times during the 4 week period (28 days), received an average of 36.1 (SD 3.2) text messages (SMS), consumed alcohol on a median of 13.0 (IQR 11.0-15.0) days, and consumed a median of 5.6 (IQR 3.3-11.8) units per drinking day in the first week, which decreased to 4.7 (IQR 2.0-6.9) units by the last week and remained active for 4.0 (IQR 3.0-4.0) weeks.

CONCLUSIONS

Personnel engaged and used the app regularly as demonstrated by the number of initializations, interactions, and time spent using InDEx. Future research is needed to evaluate the engagement with and efficacy of InDEx for the reduction of alcohol consumption and binge drinking in an armed forces population.

Online Decision Support Tool for Personalized Cancer Symptom Checking in the Community (REACT): Acceptability, Feasibility, and Usability Study

Background

Improving cancer survival in the UK, despite recent significant gains, remains a huge challenge. This can be attributed to, at least in part, patient and diagnostic delays, when patients are unaware they are suffering from a cancerous symptom and therefore do not visit a general practitioner promptly and/or when general practitioners fail to investigate the symptom or refer promptly. To raise awareness of symptoms that may potentially be indicative of underlying cancer among members of the public a symptom-based risk assessment model (developed for medical practitioner use and currently only used by some UK general practitioners) was utilized to develop a risk assessment tool to be offered to the public in community settings. Such a tool could help individuals recognize a symptom, which may potentially indicate cancer, faster and reduce the time taken to visit to their general practitioner. In this paper we report results about the design and development of the REACT (Risk Estimation for Additional Cancer Testing) website, a tool to be used in a community setting allowing users to complete an online questionnaire and obtain personalized cancer symptom-based risk estimation.

Objective

The objectives of this study are to evaluate (1) the acceptability of REACT among the public and health care practitioners, (2) the usability of the REACT website, (3) the presentation of personalized cancer risk on the website, and (4) potential approaches to adopt REACT into community health care services in the UK.

Methods

Our research consisted of multiple stages involving members of the public (n=39) and health care practitioners (n=20) in the UK. Data were collected between June 2017 and January 2018. User views were collected by (1) the “think-aloud” approach when participants using the website were asked to talk about their perceptions and feelings in relation to the website, and (2) self-reporting of website experiences through open-ended questionnaires. Data collection and data analysis continued simultaneously, allowing for website iterations between different points of data collection.

Results

The results demonstrate the need for such a tool. Participants suggest the best way to offer REACT is through a guided approach, with a health care practitioner (eg, pharmacist or National Health Service Health Check nurse) present during the process of risk evaluation. User feedback, which was generally consistent across members of public and health care practitioners, has been used to inform the development of the website. The most important aspects were: simplicity, ability to evaluate multiple cancers, content emphasizing an inviting community “feel,” use (when possible) of layperson language in the symptom screening questionnaire, and a robust and positive approach to cancer communication relying on visual risk representation both with affected individuals and the entire population at risk.

Conclusions

This study illustrates the benefits of involving public and stakeholders in developing and implementing a simple cancer symptom check tool within community. It also offers insights and design suggestions for user-friendly interfaces of similar health care Web-based services, especially those involving personalized risk estimation.

What makes online substance-use interventions engaging? A systematic review and narrative synthesis

Background

Online substance-use interventions are effective in producing reductions in harmful-use. However, low user engagement rates with online interventions reduces overall effectiveness of interventions. Identifying optimal strategies with which to engage users with online substance-use interventions may improve usage rates and subsequent effectiveness.

Objectives

(1) To identify the most prevalent engagement promoting strategies utilised to increase use of online substance-use interventions. (2) To determine whether the identified engagement promoting strategies increased said use of online substance-use interventions.

Review methods

The reviewed followed Cochrane methodology. Databases were searched for online substance-use interventions and engagement promoting strategies limited by study type (randomised controlled trial). Due to heterogeneity between engagement promoting strategies and engagement outcomes, meta-analytic techniques were not possible. Narrative synthesis methods were used.

Results

Fifteen studies were included. Five different engagement promoting strategies were identified: (1) tailoring; (2) delivery strategies; (3) incentives; (4) reminders; (5) social support. The most frequently reported engagement promoting strategies was tailoring (47% of studies), followed by reminders and social support (40% of studies) and delivery strategies (33% of studies). The narrative synthesis demonstrated that tailoring, multimedia delivery of content and reminders are potential techniques for promoting engagement. The evidence for social support was inconclusive and negative for incentives.

Conclusions

This review was the first to examine engagement promoting strategies in solely online substance-use interventions. Three strategies were identified that may be integral in promoting engagement with online substance-use interventions. However, the small number of eligible extracted studies, inconsistent reporting of engagement outcomes and diversity of engagement features prevent firmer conclusions. More high-quality trials examining engagement are required.