Design Features for Improving Mobile Health Intervention User Engagement: Systematic Review and Thematic Analysis

Background

Well-designed mobile health (mHealth) interventions support a positive user experience; however, a high rate of disengagement has been reported as a common concern regarding mHealth interventions. To address this issue, it is necessary to summarize the design features that improve user engagement based on research over the past 10 years, during which time the popularity of mHealth interventions has rapidly increased due to the use of smartphones.

Objective

The aim of this review was to answer the question “Which design features improve user engagement with mHealth interventions?” by summarizing published literature with the purpose of guiding the design of future mHealth interventions.

Methods

This review followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist. Databases, namely, PubMed, Web of Science, Cochrane Library, Ovid EMBASE, and Ovid PsycINFO, were searched for English and Chinese language papers published from January 2009 to June 2019. Thematic analysis was undertaken to assess the design features in eligible studies. The Mixed Methods Appraisal Tool was used to assess study quality.

Results

A total of 35 articles were included. The investigated mHealth interventions were mainly used in unhealthy lifestyle (n=17) and chronic disease (n=10) prevention programs. Mobile phone apps (n=24) were the most common delivery method. Qualitative (n=22) and mixed methods (n=9) designs were widely represented. We identified the following 7 themes that influenced user engagement: personalization (n=29), reinforcement (n=23), communication (n=20), navigation (n=17), credibility (n=16), message presentation (n=16), and interface aesthetics (n=7). A checklist was developed that contained these 7 design features and 29 corresponding specific implementations derived from the studies.

Conclusions

This systematic review and thematic synthesis identified useful design features that make an mHealth intervention more user friendly. We generated a checklist with evidence-based items to enable developers to use our findings easily. Future evaluations should use more robust quantitative approaches to elucidate the relationships between design features and user engagement.

Real-World Evidence of User Engagement With Mobile Health for Diabetes Management: Longitudinal Observational Study

Background

Patient support apps have risen in popularity and provide novel opportunities for self-management of diabetes. Such apps offer patients to play an active role in monitoring their condition, thereby increasing their own treatment responsibility. Although many health apps require active user engagement to be effective, there is little evidence exploring engagement with mobile health (mHealth).

Objective

This study aims to analyze the extent to which users engage with mHealth for diabetes and identify patient characteristics that are associated with engagement.

Methods

The analysis is based on real-world data obtained by Novo Nordisk’s Cornerstones4Care Powered by Glooko diabetes support app. User engagement was assessed as the number of active days and using measures expressing the persistence, longevity, and regularity of interaction within the first 180 days of use. Beta regressions were estimated to assess the associations between user characteristics and engagement outcomes for each module of the app.

Results

A total of 9051 individuals initiated use after registration and could be observed for 180 days. Among these, 55.39% (5013/9051) used the app for one specific purpose. The average user activity ratio varied from 0.05 (medication and food) to 0.55 (continuous glucose monitoring), depending on the module of the app. Average user engagement was lower if modules required manual data entries, although the initial uptake was higher for these modules. Regression analyses further revealed that although more women used the app (2075/3649, 56.86%), they engaged significantly less with it. Older people and users who were recently diagnosed tended to use the app more actively.

Conclusions

Strategies to increase or sustain the use of apps and availability of health data may target the mode of data collection and content design and should take into account privacy concerns of the users at the same time. Users’ engagement was determined by various user characteristics, indicating that particular patient groups should be targeted or assisted when integrating apps into the self-management of their disease.

Electronic Cigarette-Related Contents on Instagram: Observational Study and Exploratory Analysis

Background

Instagram is a popular social networking platform for users to upload pictures sharing their experiences. Instagram has been widely used by vaping companies and stores to promote electronic cigarettes (e-cigarettes), as well as by public health entities to communicate the risks of e-cigarette use (vaping) to the public.

Objective

We aimed to characterize current vaping-related content on Instagram through descriptive analyses.

Methods

From Instagram, 42,951 posts were collected using vaping-related hashtags in November 2019. The posts were grouped as (1) pro-vaping, (2) vaping warning, (3) neutral to vaping, and (4) not related to vaping based on the attitudes to vaping expressed within the posts. From these Instagram posts and the corresponding 18,786 unique Instagram user accounts, 200 pro-vaping and 200 vaping-warning posts as well as 200 pro-vaping and 200 vaping-warning user accounts were randomly selected for hand coding. Furthermore, follower counts and media counts of the Instagram user accounts as well as the “like” counts and hashtags of the posts were compared between pro-vaping and vaping-warning groups.

Results

There were more posts in the pro-vaping group (41,412 posts) than there were in the vaping-warning group (1539 posts). The majority of pro-vaping images were product display images (163/200, 81.5%), and the most popular image type in vaping-warning posts was educational (95/200, 47.5%). The highest proportion of pro-vaping user account type was vaping store (110/189, 58.1%), and the store account type had the highest mean number of posts (10.33 posts/account). The top 3 vaping-warning user account types were personal (79/155, 51%), vaping-warning community (37/155, 23.9%), and community (35/155, 22.6%), of which the vaping-warning community had the highest mean number of posts (3.68 posts/account). Pro-vaping user accounts had more followers (median 850) and media (median 232) than vaping-warning user accounts had (follower count: median 191; media count: 92). Pro-vaping posts had more “likes” (median 22) and hashtags (mean 20.39) than vaping-warning posts had (“like” count: median 12; hashtags: mean 7.16).

Conclusions

Instagram is dominated by pro-vaping content, and pro-vaping posts and user accounts seem to have more user engagement than vaping-warning accounts have. These results highlight the importance of regulating e-cigarette posts on social media and the urgency of identifying effective communication content and message delivery methods with the public about the health effects of e-cigarettes to ameliorate the epidemic of vaping in youth.

Optimising Web-Based Computer-Tailored Physical Activity Interventions for Prostate Cancer Survivors: A Randomised Controlled Trial Examining the Impact of Website Architecture on User Engagement

BACKGROUND

Web-based computer-tailored interventions can assist prostate cancer survivors to become more physically active by providing personally relevant behaviour change support. This study aimed to explore how changing the website architecture (free choice vs. tunnelled) impacted engagement within a physical activity computer-tailored intervention targeting prostate cancer survivors.

METHODS

On a 2:2:1 ratio, 71 Australian prostate cancer survivors with local or locally advanced disease (mean age: 66.6 years ± 9.66) were randomised into either a free-choice (N = 27), tunnelled (N = 27) or minimal intervention control arm (N =17). The primary outcome was differences in usage of the physical activity self-monitoring and feedback modules between the two intervention arms. Differences in usage of other website components between the two intervention groups were explored as secondary outcomes. Further, secondary outcomes involving comparisons between all study groups (including the control) included usability, personal relevance, and behaviour change.

RESULTS

The average number of physical activity self-monitoring and feedback modules accessed was higher in the tunnelled arm (M 2.6 SD 1.3) compared to the free-choice arm (M 1.5 SD 1.4), p = 0.01. However, free-choice participants were significantly more likely to have engaged with the social support (p = 0.008) and habit formation (p = 0.003) 'once-off' modules compared to the standard tunnelled arm. There were no other between-group differences found for any other study outcomes.

CONCLUSION

This study indicated that website architecture influences behavioural engagement. Further research is needed to examine the impact of differential usage on mechanisms of action and behaviour change.

Are You Still With Me? Continuous Engagement Assessment From a Robot's Point of View

Continuously measuring the engagement of users with a robot in a Human-Robot Interaction (HRI) setting paves the way toward in-situ reinforcement learning, improve metrics of interaction quality, and can guide interaction design and behavior optimization. However, engagement is often considered very multi-faceted and difficult to capture in a workable and generic computational model that can serve as an overall measure of engagement. Building upon the intuitive ways humans successfully can assess situation for a degree of engagement when they see it, we propose a novel regression model (utilizing CNN and LSTM networks) enabling robots to compute a single scalar engagement during interactions with humans from standard video streams, obtained from the point of view of an interacting robot. The model is based on a long-term dataset from an autonomous tour guide robot deployed in a public museum, with continuous annotation of a numeric engagement assessment by three independent coders. We show that this model not only can predict engagement very well in our own application domain but show its successful transfer to an entirely different dataset (with different tasks, environment, camera, robot and people). The trained model and the software is available to the HRI community, at https://github.com/LCAS/engagement_detector, as a tool to measure engagement in a variety of settings.

Assessment of Data Usage of Cancer e-Interventions (ADUCI) Framework for Health App Use of Cancer Patients and Their Caregivers: Framework Development Study

Background

Multimedia interventions can provide a cost-effective solution to public health needs; however, user engagement is low. Multimedia use within specific populations such as those affected by cancer differs from that of the general population. To our knowledge, there are no frameworks on how to accurately assess usage within this population to ensure that interventions are appropriate for the end users. Therefore, a framework was developed to improve the accuracy of determining data usage. Formative work included creating a data usage framework during target audience testing for smartphone app development and analysis in a pilot study.

Objective

The purpose of this study was to develop a framework for assessing smartphone app usage among people living with cancer and their caregivers.

Methods

The frequency and duration of use were compared based on manual data extraction from two previous studies and the newly developed Assessment of Data Usage of Cancer e-Interventions (ADUCI) Framework.

Results

Manual extraction demonstrated that 279 logins occurred compared with 241 when the ADUCI Framework was applied. The frequency of use in each section of the app also decreased when the ADUCI Framework was used. The total duration of use was 91,256 seconds (25.3 hours) compared with 53,074 seconds (14.7 hours) when using the ADUCI Framework. The ADUCI Framework identified 38 logins with no navigation, and there were 15 discrepancies in the data where time on a specific page of the app exceeded the login time. Practice recommendations to improve user engagement and capturing usage data include tracking data use in external websites, having a login function on apps, creating a five-star page rating functionality, using the ADUCI Framework to thoroughly clean usage data, and validating the Framework between expected and observed use.

Conclusions

Applying the ADUCI Framework may eliminate errors and allow for more accurate analysis of usage data in e-research projects. The Framework can also improve the process of capturing usage data by providing a guide for usage data analysis to facilitate evidence-based assessment of user engagement with apps.

Improving the reach of clinical practice guidelines: An experimental investigation of message framing on user engagement

Despite strong evidence for the efficacy of PTSD treatments, most affected individuals are not receiving these treatments, in part because they may not know that evidence-based treatments exist. The American Psychological Association published a website to disseminate information about their Clinical Practice Guideline for treating PTSD. In Study 1, Google Optimize was used in a field study to examine whether altering the subheadings to three of the website pages would increase site visitor engagement. On the main page and page describing treatments, no subheading alterations improved engagement. On the Patients and Families page, the subheading "say goodbye to symptoms" improved engagement on three outcome variables, including clicking a link to find a psychologist (though there were a small number of clicks). In a preregistered conceptual replication in a sample not actively seeking information about the PTSD guideline (N=578), results did not replicate. Results highlight challenges of evidence-based treatment information dissemination.

Beyond clicks and downloads: a call for a more comprehensive approach to measuring mobile-health app engagement

Downloading a mobile health (m-health) app on your smartphone does not mean you will ever use it. Telling another person about an app does not mean you like it. Using an online intervention does not mean it has had an impact on your well-being. Yet we consistently rely on downloads, clicks, 'likes' and other usage and popularity metrics to measure m-health app engagement. Doing so misses the complexity of how people perceive and use m-health apps in everyday life to manage mental health conditions. This article questions commonly used behavioural metrics of engagement in mental health research and care, and proposes a more comprehensive approach to measuring in-app engagement.

MyStrengths, a Strengths-Focused Mobile Health Tool: Participatory Design and Development

BACKGROUND

People living with chronic illnesses are an increasingly large group. Research indicates that care and self-management should not only focus on the illness and problem-oriented aspects of these individuals' lives but also support them in recognizing and leveraging their personal strengths in daily life.

OBJECTIVE

This paper presents the design and developmental process of MyStrengths, a mobile health (mHealth) app designed to help its users (people with chronic conditions) both find and make use of their personal strengths in their daily lives. Through 4 consecutive phases, this paper presents participant- and researcher-driven activities, discussions regarding design, and development of both the MyStrengths app and its content.

METHODS

During the 4 phases, we used a range of methods and activities, including (1) an idea-generating workshop aimed at creating ideas for strengths-supporting features with different stakeholders, including patients, caregivers, relatives, and designers (N=35); (2) research seminars with an international group of experts (N=6), in which the concept, theoretical background, and design ideas for the app were discussed; (3) a series of co-design workshops with people in the user group (N=22) aiming to create ideas for how to, in an engaging manner, design the app; and (4) in 4 developmental iterations, the app was evaluated by people in the user group (N=13). Content and strengths exercises were worked on and honed by the research team, the expert groups, and our internal editorial team during the entire developmental process.

RESULTS

The first phase found a wide range of stakeholder requirements to, and ideas for, strengths-focused mHealth apps. From reviewing literature during the second phase, we found a dearth of research on personal strengths with respect to people living with chronic illnesses. Activities during the third phase creatively provided numerous ideas and suggestions for engaging and gameful ways to develop and design the MyStrengths app. The final phase saw the output from all the earlier phases come together. Through multiple increasingly complete iterations of user evaluations testing and developing, the final prototype of the MyStrengths app was created.

CONCLUSIONS

Although research supports the use of strengths-focused mHealth tools to support people living with chronic illnesses, there is little guidance as to how these tools and their content should be designed. Through all activities, we found great support among participating users for strengths-focused apps, and we can consider such apps to be both appropriate and valuable. This paper illustrates how combining a range of user-, researcher-, literature-, and designer-based methods can contribute to creating mHealth tools to support people with chronic illnesses to find and use more of their own personal strengths.

Exclusively Digital Health Interventions Targeting Diet, Physical Activity, and Weight Gain in Pregnant Women: Systematic Review and Meta-Analysis

BACKGROUND

Interventions to promote a healthy diet, physical activity, and weight management during pregnancy are increasingly embracing digital technologies. Although some interventions have combined digital with interpersonal (face-to-face or telephone) delivery, others have relied exclusively on digital delivery. Exclusively digital interventions have the advantages of greater cost-effectiveness and broader reach and as such can be a valuable resource for health care providers.

OBJECTIVE

This systematic review aims to focus on exclusively digital interventions to determine their effectiveness, identify behavior change techniques (BCTs), and investigate user engagement.

METHODS

A total of 6 databases (Medical Literature Analysis and Retrieval System Online [MEDLINE], Excerpta Medica dataBASE [EMBASE], PsycINFO, Cumulated Index to Nursing and Allied Health Literature [CINAHL] Plus, Web of Science, and ProQuest) were searched for randomized controlled trials or pilot control trials of exclusively digital interventions to encourage healthy eating, physical activity, or appropriate weight gain during pregnancy. The outcome measures were gestational weight gain (GWG) and changes in physical activity and dietary behaviors. Study quality was assessed using the Cochrane Risk of Bias tool 2.0. Where possible, pooled effect sizes were calculated using a random effects meta-analysis.

RESULTS

In total, 11 studies met the inclusion criteria. The risk of bias was mostly high (n=5) or moderate (n=3). Of the 11 studies, 6 reported on GWG as the primary outcome, 4 of which also measured changes in physical activity and dietary behaviors, and 5 studies focused either on dietary behaviors only (n=2) or physical activity only (n=3). The meta-analyses showed no significant benefit of interventions on total GWG for either intention-to-treat data (-0.28 kg; 95% CI -1.43 to 0.87) or per-protocol data (-0.65 kg; 95% CI -1.98 to 0.67). Substantial heterogeneity in outcome measures of change in dietary behaviors and physical activity precluded further meta-analyses. BCT coding identified 7 BCTs that were common to all effective interventions. Effective interventions averaged over twice as many BCTs from the goals and planning, and feedback and monitoring domains as ineffective interventions. Data from the 6 studies reporting on user engagement indicated a positive association between high engagement with key BCTs and greater intervention effectiveness. Interventions using proactive messaging and feedback appeared to have higher levels of engagement.

CONCLUSIONS

In contrast to interpersonal interventions, there is little evidence of the effectiveness of exclusively digital interventions to encourage a healthy diet, physical activity, or weight management during pregnancy. In this review, effective interventions used proactive messaging, such as reminders to engage in BCTs, feedback on progress, or tips, suggesting that interactivity may drive engagement and lead to greater effectiveness. Given the benefits of cost and reach of digital interventions, further research is needed to understand how to use advancing technologies to enhance user engagement and improve effectiveness.

Surveying the Role of Analytics in Evaluating Digital Mental Health Interventions for Transition-Aged Youth: Scoping Review

BACKGROUND

Consumer-facing digital health interventions provide a promising avenue to bridge gaps in mental health care delivery. To evaluate these interventions, understanding how the target population uses a solution is critical to the overall validity and reliability of the evaluation. As a result, usage data (analytics) can provide a proxy for evaluating the engagement of a solution. However, there is paucity of guidance on how usage data or analytics should be used to assess and evaluate digital mental health interventions.

OBJECTIVE

This review aimed to examine how usage data are collected and analyzed in evaluations of mental health mobile apps for transition-aged youth (15-29 years).

METHODS

A scoping review was conducted using the Arksey and O'Malley framework. A systematic search was conducted on 5 journal databases using keywords related to usage and engagement, mental health apps, and evaluation. A total of 1784 papers from 2008 to 2019 were identified and screened to ensure that they included analytics and evaluated a mental health app for transition-aged youth. After full-text screening, 49 papers were included in the analysis.

RESULTS

Of the 49 papers included in the analysis, 40 unique digital mental health innovations were evaluated, and about 80% (39/49) of the papers were published over the past 6 years. About 80% involved a randomized controlled trial and evaluated apps with information delivery features. There were heterogeneous findings in the concept that analytics was ascribed to, with the top 3 being engagement, adherence, and acceptability. There was also a significant spread in the number of metrics collected by each study, with 35% (17/49) of the papers collecting only 1 metric and 29% (14/49) collecting 4 or more analytic metrics. The number of modules completed, the session duration, and the number of log ins were the most common usage metrics collected.

CONCLUSIONS

This review of current literature identified significant variability and heterogeneity in using analytics to evaluate digital mental health interventions for transition-aged youth. The large proportion of publications from the last 6 years suggests that user analytics is increasingly being integrated into the evaluation of these apps. Numerous gaps related to selecting appropriate and relevant metrics and defining successful or high levels of engagement have been identified for future exploration. Although long-term use or adoption is an important precursor to realizing the expected benefits of an app, few studies have examined this issue. Researchers would benefit from clarification and guidance on how to measure and analyze app usage in terms of evaluating digital mental health interventions for transition-aged youth. Given the established role of adoption in the success of health information technologies, understanding how to abstract and analyze user adoption for consumer digital mental health apps is also an emerging priority.

Applicability of the User Engagement Scale to Mobile Health: A Survey-Based Quantitative Study

BACKGROUND

There has recently been exponential growth in the development and use of health apps on mobile phones. As with most mobile apps, however, the majority of users abandon them quickly and after minimal use. One of the most critical factors for the success of a health app is how to support users' commitment to their health. Despite increased interest from researchers in mobile health, few studies have examined the measurement of user engagement with health apps.

OBJECTIVE

User engagement is a multidimensional, complex phenomenon. The aim of this study was to understand the concept of user engagement and, in particular, to demonstrate the applicability of a user engagement scale (UES) to mobile health apps.

METHODS

To determine the measurability of user engagement in a mobile health context, a UES was employed, which is a psychometric tool to measure user engagement with a digital system. This was adapted to Ada, developed by Ada Health, an artificial intelligence-powered personalized health guide that helps people understand their health. A principal component analysis (PCA) with varimax rotation was conducted on 30 items. In addition, sum scores as means of each subscale were calculated.

RESULTS

Survey data from 73 Ada users were analyzed. PCA was determined to be suitable, as verified by the sampling adequacy of Kaiser-Meyer-Olkin=0.858, a significant Bartlett test of sphericity (χ²₃₀₀=1127.1; P<.001), and communalities mostly within the 0.7 range. Although 5 items had to be removed because of low factor loadings, the results of the remaining 25 items revealed 4 attributes: perceived usability, aesthetic appeal, reward, and focused attention. Ada users showed the highest engagement level with perceived usability, with a value of 294, followed by aesthetic appeal, reward, and focused attention.

CONCLUSIONS

Although the UES was deployed in German and adapted to another digital domain, PCA yielded consistent subscales and a 4-factor structure. This indicates that user engagement with health apps can be assessed with the German version of the UES. These results can benefit related mobile health app engagement research and may be of importance to marketers and app developers.

Understanding Health Behavior Technology Engagement: Pathway to Measuring Digital Behavior Change Interventions

Researchers and practitioners of digital behavior change interventions (DBCI) use varying and, often, incongruent definitions of the term "engagement," thus leading to a lack of precision in DBCI measurement and evaluation. The objective of this paper is to propose discrete definitions for various types of user engagement and to explain why precision in the measurement of these engagement types is integral to ensuring the intervention is effective for health behavior modulation. Additionally, this paper presents a framework and practical steps for how engagement can be measured in practice and used to inform DBCI design and evaluation. The key purpose of a DBCI is to influence change in a target health behavior of a user, which may ultimately improve a health outcome. Using available literature and practice-based knowledge of DBCI, the framework conceptualizes two primary categories of engagement that must be measured in DBCI. The categories are health behavior engagement, referred to as "Big E," and DBCI engagement, referred to as "Little e." DBCI engagement is further bifurcated into two subclasses: (1) user interactions with features of the intervention designed to encourage frequency of use (ie, simple login, games, and social interactions) and make the user experience appealing, and (2) user interactions with behavior change intervention components (ie, behavior change techniques), which influence determinants of health behavior and subsequently influence health behavior. Achievement of Big E in an intervention delivered via digital means is contingent upon Little e. If users do not interact with DBCI features and enjoy the user experience, exposure to behavior change intervention components will be limited and less likely to influence the behavioral determinants that lead to health behavior engagement (Big E). Big E is also dependent upon the quality and relevance of the behavior change intervention components within the solution. Therefore, the combination of user interactions and behavior change intervention components creates Little e, which is, in turn, designed to improve Big E. The proposed framework includes a model to support measurement of DBCI that describes categories of engagement and details how features of Little e produce Big E. This framework can be applied to DBCI to support various health behaviors and outcomes and can be utilized to identify gaps in intervention efficacy and effectiveness.

Objective User Engagement With Mental Health Apps: Systematic Search and Panel-Based Usage Analysis

Background

Understanding patterns of real-world usage of mental health apps is key to maximizing their potential to increase public self-management of care. Although developer-led studies have published results on the use of mental health apps in real-world settings, no study yet has systematically examined usage patterns of a large sample of mental health apps relying on independently collected data.

Objective

Our aim is to present real-world objective data on user engagement with popular mental health apps.

Methods

A systematic engine search was conducted using Google Play to identify Android apps with 10,000 installs or more targeting anxiety, depression, or emotional well-being. Coding of apps included primary incorporated techniques and mental health focus. Behavioral data on real-world usage were obtained from a panel that provides aggregated nonpersonal information on user engagement with mobile apps.

Results

In total, 93 apps met the inclusion criteria (installs: median 100,000, IQR 90,000). The median percentage of daily active users (open rate) was 4.0% (IQR 4.7%) with a difference between trackers (median 6.3%, IQR 10.2%) and peer-support apps (median 17.0%) versus breathing exercise apps (median 1.6%, IQR 1.6%; all z≥3.42, all P<.001). Among active users, daily minutes of use were significantly higher for mindfulness/meditation (median 21.47, IQR 15.00) and peer support (median 35.08, n=2) apps than for apps incorporating other techniques (tracker, breathing exercise, psychoeducation: medians range 3.53-8.32; all z≥2.11, all P<.05). The medians of app 15-day and 30-day retention rates were 3.9% (IQR 10.3%) and 3.3% (IQR 6.2%), respectively. On day 30, peer support (median 8.9%, n=2), mindfulness/meditation (median 4.7%, IQR 6.2%), and tracker apps (median 6.1%, IQR 20.4%) had significantly higher retention rates than breathing exercise apps (median 0.0%, IQR 0.0%; all z≥2.18, all P≤.04). The pattern of daily use presented a descriptive peak toward the evening for apps incorporating most techniques (tracker, psychoeducation, and peer support) except mindfulness/meditation, which exhibited two peaks (morning and night).

Conclusions

Although the number of app installs and daily active minutes of use may seem high, only a small portion of users actually used the apps for a long period of time. More studies using different datasets are needed to understand this phenomenon and the ways in which users self-manage their condition in real-world settings.

Association Between User Engagement of a Mobile Health App for Gout and Improvements in Self-Care Behaviors: Randomized Controlled Trial

Background

Mobile health (mHealth) apps represent a promising approach for improving health outcomes in patients with chronic illness, but surprisingly few mHealth interventions have investigated the association between user engagement and health outcomes. We aimed to examine the efficacy of a recommended, commercially available gout self-management app for improving self-care behaviors and to assess self-reported user engagement of the app in a sample of adults with gout.

Objective

Our objective was to examine differences in self-reported user engagement between a recommended gout app (treatment group) and a dietary app (active control group) over 2 weeks as well as to examine any differences in self-care behaviors and illness perceptions.

Methods

Seventy-two adults with gout were recruited from the community and three primary and secondary clinics. Participants were randomized to use either Gout Central (n=36), a self-management app, or the Dietary Approaches to Stop Hypertension Diet Plan (n=36), an app based on a diet developed for hypertension, for 2 weeks. The user version of the Mobile Application Rating Scale (uMARS, scale: 1 to 5) was used after the 2 weeks to assess self-reported user engagement, which included an open-ended question. Participants also completed a self-report questionnaire on self-care behaviors (scale: 1-5 for medication adherence and diet and 0-7 for exercise) and illness perceptions (scale: 0-10) at baseline and after the 2-week trial. Independent samples t tests and analysis of covariance were used to examine differences between groups at baseline and postintervention.

Results

Participants rated the gout app as more engaging (mean difference –0.58, 95% CI –0.96 to –0.21) and more informative (mean difference –0.34, 95% CI –0.67 to –0.01) than the dietary app at the 2-week follow-up. The gout app group also reported a higher awareness of the importance of gout (mean difference –0.64, 95% CI –1.27 to –0.003) and higher knowledge/understanding of gout (mean difference –0.70, 95% CI –1.30 to –0.09) than the diet app group at follow-up. There were no significant differences in self-care behaviors between the two groups postintervention. The gout app group also demonstrated stronger negative beliefs regarding the impact of gout (mean difference –2.43, 95% CI –3.68 to –1.18), stronger beliefs regarding the severity of symptoms (mean difference –1.97, 95% CI –3.12 to –0.82), and a stronger emotional response to gout (mean difference –2.38, 95% CI –3.85 to –0.90) at follow-up. Participant feedback highlighted the importance of tracking health-related information, customizing to the target group/individual, providing more interactive features, and simplifying information.

Conclusions

Participants found the commercially available gout app more engaging. However, these findings did not translate into differences in self-care behaviors. The gout app group also demonstrated stronger negative illness perceptions at the follow-up. Overall, these findings suggest that the development of gout apps would benefit from a user-centered approach with a focus on daily, long-term self-care behaviors as well as modifying illness beliefs.

Trial Registration

Australian New Zealand Clinical Trials Registry ACTRN12617001052325; https://anzctr.org.au/Trial/Registration/TrialReview.aspx?id=373217.

Factors Influencing User Engagement of Health Information Disseminated by Chinese Provincial Centers for Disease Control and Prevention on WeChat: Observational Study

Background

Social media is currently becoming a new channel for information acquisition and exchange. In China, with the growing popularity of WeChat and WeChat official accounts (WOAs), health promotion agencies have an opportunity to use them for successful information distribution and diffusion online.

Objective

We aimed to identify features of articles pushed by WOAs of Chinese provincial Centers for Disease Control and Prevention (CDC) that are associated with user engagement.

Methods

We searched and subscribed to 28 WOAs of provincial CDCs. Data for this study consisted of WeChat articles on these WOAs between January 1, 2017 and December 31, 2017. We developed a features frame containing title type, article content, article type, communication skills, number of marketing elements, and article length for each article and coded the data quantitatively using a coding scheme that assigned numeric values to article features. We examined the descriptive characteristics of articles for every WOA and generated descriptive statistics for six article features. The amount of reading and liking was converted into the level of reading and liking by the 75% position. Two-category univariate logistic regression and multivariable logistic regression were conducted to explore associations between the features of the articles and user engagement, operationalized as reading level and liking level.

Results

All provincial CDC WOAs provided a total of 5976 articles in 2017. Shanghai CDC articles attracted the most user engagement, and Ningxia CDC articles attracted the least. For all articles, the median reading was 551.5 and the median liking was 10. Multivariable logistic regression analysis revealed that article content, article type, communication skills, number of marketing elements, and article length were associated with reading level and liking level. However, title type was only associated with liking level.

Conclusions

How social media can be used to best achieve health information dissemination and public health outcomes is a topic of much discussion and study in the public health community. Given the lack of related studies based on WeChat or official accounts, we conducted this study and found that article content, article type, communication skills, number of marketing elements, article length, and title type were associated with user engagement. Our study may provide public health and community leaders with insight into the diffusion of important health topics of concern.

Examining Predictors of Real-World User Engagement with Self-Guided eHealth Interventions: Analysis of Mobile Apps and Websites Using a Novel Dataset

Background

The literature suggests that the product design of self-guided electronic health (eHealth) interventions impacts user engagement. Traditional trial settings, however, do not enable the examination of these relationships in real-world use.

Objective

This study aimed to examine whether the qualities of product design, research evidence, and publicly available data predict real-world user engagement with mobile and Web-based self-guided eHealth interventions.

Methods

This analysis included self-guided mobile and Web-based eHealth interventions available to the public—with their qualities assessed using the Enlight suite of scales. Scales included Usability, Visual Design, User Engagement, Content, Therapeutic Persuasiveness, Therapeutic Alliance, Credibility, and Research Evidence. Behavioral data on real-world usage were obtained from a panel that provides aggregated nonpersonal information on user engagement with websites and mobile apps, based on a time window of 18 months that was set between November 1, 2016 and April 30, 2018. Real-world user engagement variables included average usage time (for both mobile apps and websites) and mobile app user retention 30 days after download.

Results

The analysis included 52 mobile apps (downloads median 38,600; interquartile range [IQR] 116,000) and 32 websites (monthly unique visitors median 5689; IQR 30,038). Results point to moderate correlations between Therapeutic Persuasiveness, Therapeutic Alliance, and the 3 user engagement variables (.31≤rs≤.51; Ps≤.03). Visual Design, User Engagement, and Content demonstrated similar degrees of correlation with mobile app engagement variables (.25≤rs≤.49; Ps≤.04) but not with average usage time of Web-based interventions. Positive correlations were also found between the number of reviews on Google Play and average app usage time (r=.58; P<.001) and user retention after 30 days (r=.23; P=.049). Although several product quality ratings were positively correlated with research evidence, the latter was not significantly correlated with real-world user engagement. Hierarchical stepwise regression analysis revealed that either Therapeutic Persuasiveness or Therapeutic Alliance explained 15% to 26% of user engagement variance. Data on Google Play (number of reviews) explained 15% of the variance of mobile app usage time above Enlight ratings; however, publicly available data did not significantly contribute to explaining the variance of the other 2 user-engagement variables.

Conclusions

Results indicate that the qualities of product design predict real-world user engagement with eHealth interventions. The use of real-world behavioral datasets is a novel way to learn about user behaviors, creating new avenues for eHealth intervention research.

Insights From a Text Messaging-Based Sexual and Reproductive Health Information Program in Tanzania (m4RH): Retrospective Analysis

BACKGROUND

Many mobile health (mHealth) interventions have the potential to generate and store vast amounts of system-generated participant interaction data that could provide insight into user engagement, programmatic strengths, and areas that need improvement to maximize efficacy. However, despite the popularity of mHealth interventions, there is little documentation on how to use these data to monitor and improve programming or to evaluate impact.

OBJECTIVE

This study aimed to better understand how users of the Mobile for Reproductive Health (m4RH) mHealth intervention engaged with the program in Tanzania from September 2013 to August 2016.

METHODS

We conducted secondary data analysis of longitudinal data captured by system logs of participant interactions with the m4RH program from 127 districts in Tanzania from September 2013 to August 2016. Data cleaning and analysis was conducted using Stata 13. The data were examined for completeness and "correctness." No missing data was imputed; respondents with missing or incorrect values were dropped from the analyses.

RESULTS

The total population for analysis included 3,673,702 queries among 409,768 unique visitors. New users represented roughly 11.15% (409,768/3,673,702) of all queries. Among all system queries for new users, 46.10% (188,904/409,768) users accessed the m4RH main menu. Among these users, 89.58% (169,218/188,904) accessed specific m4RH content on family planning, contraceptive methods, adolescent-specific and youth-specific information, and clinic locations after first accessing the m4RH main menu. The majority of these users (216,422/409,768, 52.82%) requested information on contraceptive methods; fewer users (23,236/409,768, 5.67%) requested information on clinic location. The conversion rate was highest during the first and second years of the program when nearly all users (11,246/11,470, 98.05%, and 33,551/34,830, 96.33%, respectively) who accessed m4RH continued on to query more specific content from the system. The rate of users that accessed m4RH and became active users declined slightly from 98.05% (11,246/11,470) in 2013 to 87.54% (56,696/64,765) in 2016. Overall, slightly more than one-third of all new users accessing m4RH sent queries at least once per month for 2 or more months, and 67.86% (278,088/409,768) of new and returning users requested information multiple times per month. Promotional periods were present for 15 of 36 months during the study period.

CONCLUSIONS

The analysis of the rich data captured provides a useful framework with which to measure the degree and nature of user engagement utilizing routine system-generated data. It also contributes to knowledge of how users engage with text messaging (short message service)-based health promotion interventions and demonstrates how data generated on user interactions could inform improvements to the design and delivery of a service, thereby enhancing its effectiveness.

The Effects of User Engagements for User and Company Generated Videos on Music Sales: Empirical Evidence From YouTube

With the growth of social network services, users have been able to freely create and share music in ways that were once thought unimaginable. Sharing a music video through such platforms can now be done simply by anyone with access to a computer or smart phone. Online music content can be divided into two major categories—user-generated content (UGC) and company-generated content (CGC). While previous studies on the content of online music videos have examined the impact of this content on, for example, a company's marketing effectiveness and brand image, they have given little attention to the different effects of user engagement of UGC and user engagement of CGC. This study attempts to address this lack by examining how these two kinds of user engagement differently influence user music choices. We will also examine the differing levels of impact on users across the initial, middle, and final periods after a song is released. In order to examine the different impacts of user engagement, we apply an estimation of generalized least squares (GLS) with a panel dataset of 1,035 songs found on Gaon, the official South Korean music ranking chart. We use the number of music video shares generated by users and firms as proxy variables of user engagement in UGC and CGC, respectively. We find that user engagement of UGC and CGC positively influence music sales. We also find that the effects are not static, but rather change in the initial, middle, and final periods after a song is released. In particular, this study finds that the effect of user engagement of UGC on sales is greater than the effect of user engagement of CGC in the initial period but that the effect of the latter becomes similar to that of former at the end of this period. This finding suggests that managers of digital music providers should encourage consumers to create their own content in the initial period, as potential consumers are more likely to buy songs with more UGC content and shares in the initial period.

Content-Sensitive Characterization of Peer Interactions of Highly Engaged Users in an Online Community for Smoking Cessation: Mixed-Methods Approach for Modeling User Engagement in Health Promotion Interventions

Background

Online communities provide affordable venues for behavior change. However, active user engagement holds the key to the success of these platforms. In order to enhance user engagement and in turn, health outcomes, it is essential to offer targeted interventional and informational support.

Objective

In this paper, we describe a content plus frequency framework to enable the characterization of highly engaged users in online communities and study theoretical techniques employed by these users through analysis of exchanged communication.

Methods

We applied the proposed methodology for analysis of peer interactions within QuitNet, an online community for smoking cessation. Firstly, we identified 144 highly engaged users based on communication frequency within QuitNet over a period of 16 years. Secondly, we used the taxonomy of behavior change techniques, text analysis methods from distributional semantics, machine learning, and sentiment analysis to assign theory-driven labels to content. Finally, we extracted content-specific insights from peer interactions (n=159,483 messages) among highly engaged QuitNet users.

Results

Studying user engagement using our proposed framework led to the definition of 3 user categories—conversation initiators, conversation attractors, and frequent posters. Specific behavior change techniques employed by top tier users (threshold set at top 3) within these 3 user groups were found to be goal setting, social support, rewards and threat, and comparison of outcomes. Engagement-specific trends within sentiment manifestations were also identified.

Conclusions

Use of content-inclusive analytics has offered deep insight into specific behavior change techniques employed by highly engaged users within QuitNet. Implications for personalization and active user engagement are discussed.

Do Monetary Incentives Influence Users' Behavior in Participatory Sensing?

Participatory sensing combines the powerful sensing capabilities of current mobile devices with the mobility and intelligence of human beings, and as such has to potential to collect various types of information at a high spatial and temporal resolution. Success, however, entirely relies on the willingness and motivation of the users to carry out sensing tasks, and thus it is essential to incentivize the users’ active participation. In this article, we first present an open, generic participatory sensing framework (Citizense) which aims to make participatory sensing more accessible, flexible and transparent. Within the context of this framework we adopt three monetary incentive mechanisms which prioritize the fairness for the users while maintaining their simplicity and portability: fixed micro-payment, variable micro-payment and lottery. This incentive-enabled framework is then deployed on a large scale, real-world case study, where 230 participants were exposed to 44 different sensing campaigns. By randomly distributing incentive mechanisms among participants and a subset of campaigns, we study the behaviors of the overall population as well as the behaviors of different subgroups divided by demographic information with respect to the various incentive mechanisms. As a result of our study, we can conclude that (1) in general, monetary incentives work to improve participation rate; (2) for the overall population, a general descending order in terms of effectiveness of the incentive mechanisms can be established: fixed micro-payment first, then lottery-style payout and finally variable micro-payment. These two conclusions hold for all the demographic subgroups, even though different different internal distances between the incentive mechanisms are observed for different subgroups. Finally, a negative correlation between age and participation rate was found: older participants contribute less compared to their younger peers.

Predictors of Facebook User Engagement With Health-Related Content for Gay, Bisexual, and Other Men Who Have Sex With Men: Content Analysis

Background

Social media is used by community-based organizations (CBOs) to promote the well-being of gay and bisexual men (GBM). However, few studies have quantified which factors facilitate the diffusion of health content tailored for sexual minorities.

Objective

The aim of this study was to identify post characteristics that can be leveraged to optimize the health promotion efforts of CBOs on Facebook.

Methods

The Facebook application programming interface was used to collect 5 years’ of posts shared across 10 Facebook pages administered by Vancouver-based CBOs promoting GBM health. Network analysis assessed basic indicators of network structure. Content analyses were conducted using informatics-based approaches. Hierarchical negative binomial regression of post engagement data was used to identify meaningful covariates of engagement.

Results

In total, 14,071 posts were shared and 21,537 users engaged with these posts. Most users (n=13,315) engaged only once. There was moderate correlation between the number of posts and the number of CBOs users engaged with (r=.53, P<.001). Higher user engagement was positively associated with positive sentiment, sharing multimedia, and posting about pre-exposure prophylaxis, stigma, and mental health. Engagement was negatively associated with asking questions, posting about dating, and sharing posts during or after work (versus before).

Conclusions

Results highlight the existence of a core group of Facebook users who facilitate diffusion. Factors associated with greater user engagement present CBOs with a number of strategies for improving the diffusion of health content.

Motivational Factors for User Engagement with mHealth Apps

The widespread adoption of smartphones creates an enormous potential to improve healthcare services. Numerous apps, sensors, and devices are developed for health self-management purposes. However, adoption rates remain low and long-term user engagement is a major issue. The goal of this study is to identify major motivational factors that can facilitate prolonged use of mobile health systems. To this end, we conducted 16 interviews with representatives of various cultural backgrounds, disease history, age, and gender. Participants' experiences indicated that existing systems were unable to answer their self-management needs properly. People with a disease history favored learning from data, as well as from others via social media integration. People without chronic disease felt more reserved about social media integration. In conclusion, systems that collect and share personal data should have a clear opt-in or opt-out option to motivate usage. Additionally, researchers and mobile health system developers could achieve long-term adoption by giving clear answers to privacy and trust issues, while offering people strong added value according to their individual needs.

Service user engagement in health service reconfiguration: a rapid evidence synthesis

OBJECTIVE

To assess what is known about effective patient and public engagement in health service reconfiguration processes and identify implications for further research and health care practice.

METHODS

Rapid systematic review of published and grey literature to identify methods or approaches to engagement in decisions about health service reconfiguration; and to examine how engagement has worked or not worked in specific examples of system change. Following a search for literature published in English from 2000 to March 2014, eight systematic reviews, seven primary studies and 24 case studies (of which 6 were exemplars) were included. We undertook a narrative synthesis to consider five aspects of engagement with health service reconfiguration.

RESULTS

Engagement varied in nature and intensity, and efforts generally involved multiple methods. There was no evidence on the isolated impact of any particular engagement method or collection of methods. In general, engagement was most likely to be successful when started early, when led and supported by clinicians, and when it offered opportunities for genuine interaction. The impact of engagement was variably measured and demonstrated, and frequently defined as process measures rather than the outcomes of proposals for service reconfiguration. Little was reported on the potential negative impact of service user engagement.

CONCLUSIONS

Patients and the public can be engaged through various methods. Problems often arise because decision-makers paid insufficient attention to issues considered important by patients and the public. Guidance setting out the stages of reconfiguration and opportunities for service user input could be a helpful practical framework for future engagement activity. Future evaluation and explicit reporting of engagement and impact is needed.

An 8-week web-based weight loss challenge with celebrity endorsement and enhanced social support: observational study

Background

Initial engagement and weight loss within Web-based weight loss programs may predict long-term success. The integration of persuasive Web-based features may boost engagement and therefore weight loss.

Objective

To determine whether an 8-week challenge within a commercial Web-based weight loss program influenced weight loss, website use, and attrition in the short term, when compared to the standard program.

Methods

De-identified data for participants (mean age 36.7±10.3 years; 86% female) who enrolled in the Biggest Loser Club (BLC) (n=952) and the BLC’s Shannan Ponton Fast Track Challenge (SC) for 8 weeks (n=381) were compared. The BLC program used standard evidence-based website features, with individualized calorie and exercise targets to facilitate a weight loss of 0.5-1 kg per week (–500kcal/day less than estimated energy expenditure). SC used the same website features but in addition promoted greater initial weight loss using a 1200 kcal/day energy intake target and physical activity energy expenditure of 600 kcal/day. SC used persuasive features to facilitate greater user engagement, including offering additional opportunities for social support (eg, webinar meetings with a celebrity personal trainer and social networking) endorsed by a celebrity personal trainer. Self-reported weekly weight records were used to determine weight change after 8 weeks. A primary analysis was undertaken using a generalized linear mixed model (GLMM) with all available weight records for all participants included. Dropout (participants who cancelled their subscription) and nonusage (participants who stopped using the Web-based features) attrition rates at 8 weeks were calculated. The number of participants who accessed each website feature and the total number of days each feature was used were calculated. The difference between attrition rates and website use for the two programs were tested using chi-square and Wilcoxon Rank Sum tests, respectively.

Results

Using GLMM, including weight data for all participants, there was significantly greater (P=.03) 8-week weight loss in SC (–5.1 kg [–5.5 to –4.6 kg] or –6.0%) compared to BLC participants (–4.5 kg [–4.8, –4.2] or –5.0%). Dropout rates were low and consistent across groups (BLC: 17 (1.8%) vs SC: 2 (0.5%), P=.08) and 48.7% (456/936) of BLC and 51.2% (184/379) of SC participants accessed the website at 8 weeks, with no difference between programs (P=.48). SC participants accessed the discussion forums, menu plans, exercise plans, and educational materials significantly more than BLC participants (P<.05).

Conclusions

Using a short-term challenge with persuasive features, including online social support with endorsement by a celebrity personal trainer, as well as a greater energy balance deficit, within a commercial Web-based weight loss program may facilitate greater initial weight loss and engagement with some program components. The results support the need for a more rigorous and prospective evaluation of Web-based weight loss programs that incorporate additional strategies to enhance initial weight loss and engagement, such as a short-term challenge.