Investigating the Direct Impact of a Gamified Versus Nongamified Well-Being Intervention: An Exploratory Experiment

The conceptualisation and measurement of engagement in digital health

Digital tools are an increasingly important component of healthcare, but their potential impact is commonly limited by a lack of user engagement. Digital health evaluations of engagement are often restricted to system usage metrics, which cannot capture a full understanding of how and why users engage with an intervention. This study aimed to examine how theory-based, multifaceted measures of engagement with digital health interventions capture different components of engagement (affective, cognitive, behavioural, micro, and macro) and to consider areas that are unclear or missing in their measurement. We identified and compared two recently developed measures that met these criteria (the Digital Behaviour Change Intervention Engagement Scale and the TWente Engagement with Ehealth Technologies Scale). Despite having similar theoretical bases and being relatively strongly correlated, there are key differences in how these scales aim to capture engagement. We discuss the implications of our analysis for how affective, cognitive, and behavioural components of engagement can be conceptualised and whether there is value in distinguishing between them. We conclude with recommendations for the circumstances in which each scale may be most useful and for how future measure development could supplement existing scales.

The "STAR WARS: The First Jedi" Program-Effects of Gamification on Psychological Well-Being of College Students

Background: College students are particularly susceptible to developing mental health problems. Gamification can be a powerful teaching strategy to favor motivation and behavioral changes for the improvement of psychological well-being. Purpose: To analyze the effect of a 14-week gamification-based physical education teaching program on psychological well-being of college students. Materials and Methods: A total of 112 college students (21.22 ± 2.55 years) were recruited and assigned to a gamification-based group or a control group. The intervention group participated in a gamification-based program under the framework of "STAR WARS" science fiction sage. Participants from this group used a mobile app with a countdown timer and the way for them to gain lifetime was by doing physical activity (PA) and formative challenges. The control group did not experience the gamification approach itself as their classes were based on traditional methodology. They were asked to maintain their usual lifestyle and were not required to do PA. Emotional intelligence, personal initiative, entrepreneurial attitude, resilience, and self-efficacy were self-reported. Results: The gamification-based group showed a significantly higher improvement of emotional intelligence (P ≤ 0.006), personal initiative (P ≤ 0.033), entrepreneurial attitude (P < 0.001), resilience (P = 0.002), and self-efficacy (P < 0.001), compared with the control group. Overall effect sizes were of medium magnitude for emotional intelligence and personal initiative, large magnitude for entrepreneurial attitude, and small magnitude for resilience and self-efficacy. Conclusion: Gamification and the use of mobile apps may become powerful teaching strategies to motivate college students toward healthier lifestyle behaviors that have psychological well-being benefits.

Promoting Physical and Mental Health among Children and Adolescents via Gamification-A Conceptual Systematic Review

The rapid growth in digital technology usage among children and adolescents has highlighted the need for novel approaches to promote their physical and mental health. This paper investigates the viability of gamification-the application of game mechanics to non-gaming contexts-as a potent instrument for health promotion and mental health support. This conceptual systematic review seeks to examine the various published articles promoting children and adolescents' physical and mental health through gamified techniques. These interventions can provide an interactive and engaging platform for encouraging physical activity, promoting healthy nutrition, enhancing emotional regulation, and promoting mental health. The significance of this topic stems from the pervasive use of electronic games, beginning at a young age, which makes them popular educational tools. For the review to be systematic and reproducible, the PsycINFO, Scopus, PubMed, and Elsevier databases were searched and the PRISMA method was utilized for the analysis. After analyzing the research data, empirical studies assessing the use of gamification in promoting adolescents' physical and mental health are discussed. In conclusion, gamification has demonstrated promise for promoting children's and adolescents' physical and mental health. It improves motivation, commitment, and adherence to healthy behaviors. However, additional research is required to evaluate gamification interventions' long-term effectiveness and sustainability in promoting health behaviors among this population.

Effectiveness of gamified digital interventions in mental health prevention and health promotion among adults: a scoping review

BACKGROUND

Though still a young field of research, gamified digital interventions have demonstrated potential in exerting a favourable impact on health and overall well-being. With the increasing use of the internet and digital devices, the integration of game elements presents novel opportunities for preventing mental disorders and enhancing mental health. Hence, this review aims to assess the effectiveness of gamified interventions focusing on preventing mental disorders or promoting mental health among adults.

METHODS

Based on a scoping review across four databases (MEDLINE, Embase, PsycInfo and Web of Science), 7,953 studies were initially identified. After removing duplicates and screening titles, abstracts and full texts, 16 studies were identified as suitable for inclusion in a narrative synthesis of findings. We included interventional studies encompassing an intervention and a control group aiming to investigate the effectiveness of the use of gamified digital mental health interventions and the use of gamified digital elements.

RESULTS

Overall, positive effects of gamified interventions on mental health-related outcomes were identified. In particular, beneficial consequences for psychological well-being and depressive symptoms were observed in all studies. However, further outcomes, such as resilience, anxiety, stress or satisfaction with life, showed heterogenous findings. Most game elements used were reward, sensation and progress, whilst the quantity of elements was not consistent and, therefore, no substantiated conclusion regarding the (optimal) quantity or composition of game elements can be drawn. Further, the outcomes, measurements and analyses differed greatly between the 16 included studies making comparisons difficult.

CONCLUSION

In summary, this review demonstrates the potential of integrating digital game elements on mental health and well-being with still a great gap of research. A taxonomy is needed to adequately address relevant game elements in the field of mental health promotion and prevention of mental disorders. Therefore, future studies should explicitly focus on the mechanisms of effect and apply rigorous study designs.

What does it mean to be engaged with digital health interventions? A qualitative study into the experiences of engaged users and the views of professionals

Objective

Digital health interventions (DHIs) hold promise for influencing health behaviors positively, but their widespread implementation and effectiveness remain limited. Engagement is crucial for DHI effectiveness, yet its conceptualization is debated. This qualitative study explores engagement from user and professional perspectives.

Methods

Twenty self-proclaimed engaged health app users participated in semistructured interviews, and 13 professionals working with DHIs completed an online survey.

Results

Interviews with health app users revealed three key components of their sense of engagement: behavioral, cognitive, and affective. Behavioral engagement includes routine, effortless, and dynamic usage; emphasizing the importance of the quality of fit between user and technology over frequency of use. Cognitive engagement encompasses the technology's utility as a tool for supporting behavior change, providing new insights, and enhancing motivation. Affective engagement involves enjoying progress, deriving pleasure from using the technology, and identifying with the technology. Notably, participants exhibited varying emphasis on these components. Professionals, in a parallel inquiry, agreed on the relevance of behavior, cognition, and affect in defining engagement. In their understanding, behavioral engagement is often associated with adherence and frequency of use, while cognitive engagement emphasizes understanding, motivation, and achieving cognitive outcomes. Affective engagement, although diverse, is recognized as a critical dimension. In addition, it was noticeable that users and professionals perceived microengagement (with the DHI) and macroengagement (with the target behavior) as interconnected.

Conclusion

To conclude, this study contributes a nuanced understanding of the multifaceted nature of engagement, informing future measurement of the concept, DHI design, and implementation strategies for improved user experiences and outcomes.

Experiences of Users with an Online Self-Guided Mental Health Training Program Using Gamification

Young adulthood is a period of high risk for the development of mental health concerns. Increasing well-being among young adults is important to prevent mental health concerns and their consequences. Self-compassion has been identified as a modifiable trait with the potential to protect against mental health concerns. An online self-guided mental health training program using gamification was developed and the user experience was evaluated in a 6-week experimental design. During this period, 294 participants were allocated to use the online training program via a website. User experience was assessed via self-report questionnaires, and interaction data for the training program were also collected. Results showed that those who completed the intervention (n= 47) visited the website on average 3.2 days a week, with a mean of 45.8 interactions during the 6 weeks. Participants report positive user experiences of the online training, on average a System Usability Scale Brooke (1) score of 79.1 (out of 100) at the end-point. Participants showed positive engagement with story elements of the training, based on an average score of 4.1 (out of 5) in the evaluation of the story at the end-point. This study found the online self-compassion intervention for youth to be acceptable, although some features seem preferred by users as compared to others. Gamification in the form of a guiding story and a reward structure seemed to be a promising element for successfully motivating participants and serving as a guiding metaphor for self-compassion.

Factors associated with young adult engagement with a web-based sickle cell reproductive health intervention

Objective

To determine the factors predicting the engagement of young adults who have sickle cell disease (SCD) or sickle cell trait (SCT) with an online reproductive health education intervention and engagement effects on knowledge.

Methods

The cross-sectional study included 167 participants who completed the web-based intervention either face-to-face (F2F) or online delivery (OL). Measures include: time used relative to length of the intervention narration and media (engagement) and the SCKnowIQ questionnaire. Ordinal regression was conducted.

Results

The sample mean age was 26-years (SD=5), 68% were female, 54% had SCD, and 68% were in the F2F group. Adjusting for age, partner sickle cell status, marital status, and education, participants who were female (p=.003), had SCD (p=.018), or had F2F delivery (p < .001) were more likely to spend more time on the intervention. Adjusting for baseline knowledge and modality, more time spent on the intervention was associated with higher posttest knowledge (p=.006).

Conclusions

Future studies are necessary to understand reasons underpinning engagement and to investigate other unmeasured factors, such as intervention interactivity elements, that could also be associated with engagement.

Innovation

This study of young adults with SCD or SCT provides much needed insight about their engagement with online reproductive health education.

Gaming My Way to Recovery: A Systematic Scoping Review of Digital Game Interventions for Young People's Mental Health Treatment and Promotion

Nearly all young people use the internet daily. Many youth with mental health concerns, especially since the Covid-19 pandemic, are using this route to seek help, whether through digital mental health treatment, illness prevention tools, or supports for mental wellbeing. Videogames also have wide appeal among young people, including those who receive mental health services. This review identifies the literature on videogame interventions for young people, ages 12-29, and maps the data on game use by those with mental health and substance use problems, focusing on evidence for the capacity of games to support treatment in youth mental health services; how stakeholders are involved in developing or evaluating games; and any potential harms and ethical remedies identified. A systematic scoping review methodology was used to identify and assess relevant studies. A search of multiple databases identified a total of 8,733 articles. They were screened, and 49 studies testing 32 digital games retained. An adapted stepped care model, including four levels, or steps, based on illness manifestation and severity, was used as a conceptual framework for organizing target populations, mental health conditions and corresponding digital games, and study results. The 49 selected studies included: 10 studies (20.4%) on mental health promotion/prevention or education for undiagnosed youth (Step 0: 7 games); 6 studies (12.2%) on at-risk groups or suspected mental problems (Step 1: 5 games); 24 studies (49.0%) on mild to moderate mental conditions (Steps 2-3: 16 games); and 9 studies (18.4%) focused on severe and complex mental conditions (Step 4: 7 games). Two interventions were played by youth at more than one level of illness severity: the SPARX game (Steps 1, 2-3, 4) and Dojo (Steps 2-3 and 4), bringing the total game count to 35 with these repetitions. Findings support the potential integration of digital games in youth services based on study outcomes, user satisfaction, relatively high program retention rates and the potential usefulness of most games for mental health treatment or promotion/prevention. Most studies included stakeholder feedback, and involvement ratings were very high for seven games. Potential harms were not addressed in this body of research. This review provides an important initial repository and evaluation of videogames for use in clinical settings concerned with youth mental health.

Determining Priorities in the Aboriginal and Islander Mental Health Initiative for Youth App Second Phase Participatory Design Project: Qualitative Study and Narrative Literature Review

Background

Digital mental health tools can promote access to culturally safe early intervention mental health services for Aboriginal and Torres Strait Islander young people. Participatory design methodology facilitates user engagement in the co-design of digital resources. However, several challenges have been identified that limit the methodological rigor of this approach.

Objective

This paper aims to present an in-depth account of the second phase of participatory design in the development of the Aboriginal and Islander Mental Health Initiative for Youth (AIMhi-Y) app.

Methods

A first idea storyboard, generated from a formative phase of the AIMhi-Y project, was refined through a series of youth co-design workshops and meetings. A narrative review of the literature, 6 service provider interviews, and engagement with an expert reference group also informed the design process. Generative design activities, storyboarding, discussions, and voting strategies were used.

Results

The participatory design process identified the app features preferred by young people and service providers and assessed their alignment with current recommendations from the scientific literature. Findings from the co-design process are presented across 9 app characteristic domains. Integration of findings into app design proved complex. Although most preferred features identified by young people were included to some degree, other inclusions were restricted by budget, time, and the need to integrate best practice recommendations. A process of prioritization was required.

Conclusions

Participatory design is often cited in the development of digital mental health resources; however, methods are diverse and often lack detailed descriptions. This study reports the outcomes and strategies used to determine priorities in the second phase of the development of the AIMhi-Y app. We provide an example and the key learnings to inform others seeking to use participatory design with a similar cohort.

Game-Based Dual-Task Exercise Program for Children with Cerebral Palsy: Blending Balance, Visuomotor and Cognitive Training: Feasibility Randomized Control Trial

The objective of this exploratory randomized controlled trial (RCT) was to provide evidence for the feasibility and therapeutic value of a novel game-based dual-task balance exercise program in children with cerebral palsy (CP). Twenty children with CP were recruited and randomized into two groups: (a) the conventional balance training group (CG) and (b) the experimental group (XG), which received a game-based dual-task (DT) balance exercise program. Both groups received their respective therapy programs for 12 weeks at a frequency of three sessions per week. Semi-structured interviews with the parents and children and qualitative analysis were conducted to evaluate the children's experiences with the game-based exercise program. The quantitative analysis included (a) the Pediatric Balance Scale (PBS), (b) Gross Motor Function Measure-88 (GMFM-88), and (c) computerized measures of standing balance performance during various dual-task conditions. Compliance was 100% for all 20 participants. Four themes captured the range of each participant's experiences and opinions: (a) reasons for participation, (b) likes and dislikes with the technologies, (c) positive effects of the program, and (d) future expectations. Children in the XG demonstrated greater improvements in PBS, GMFM, and DT balance measures as compared to children in the CG. The findings demonstrate feasible trial procedures and acceptable DT-oriented training with a high compliance rate and positive outcomes. These findings support further research and development and progression to the next phase of a full-scale RCT to evaluate the clinical effectiveness of the game-based DT balance exercise program for children with CP.

Evaluation of a Web-Based Stress Management Program for Persons Experiencing Work-Related Stress in Sweden (My Stress Control): Randomized Controlled Trial

BACKGROUND

Stress is one of the most common reasons for sick leave. Web-based interventions have the potential to reach an unlimited number of users at a low cost and have been shown to be effective in addressing several health-related problems. Handling stress on an individual level is related to behavior change. To support behavioral changes in stress management, My Stress Control (MSC) was developed. The development of MSC was based on several health psychology theories and models; however, central in the development were Social Cognitive Theory, Theory of Reasoned Action, Theory of Planned Behavior, Transactional Theory of Stress and Coping, and the Transtheoretical Model and Stages of Change. MSC is a fully automated program. The program is tailored to the user's specific needs for stress management and behavior change.

OBJECTIVE

In this study, we aim to conduct a randomized controlled trial to evaluate the extent to which MSC affects perceived stress in persons experiencing work-related stress.

METHODS

This was a randomized controlled trial with 2 arms. Study participants were recruited by visiting the worksites and workplace meetings. Participants were assigned to the intervention or wait-list group. Web-based questionnaires were used before and after the intervention to collect data. Perceived stress measured using the Perceived Stress Scale-14 was the primary outcome measurement. Analyses were conducted for both between-group and within-group changes.

RESULTS

A total of 92 participants were included in this study: 48 (52%) in the intervention group and 44 (48%) in the wait-list group. Overall, 25% (12/48) of participants in the intervention group and 43% (19/44) of participants in the wait-list group completed the postintervention assessment. There were no significant effects on perceived stress between the intervention and wait-list groups or within the groups. A small effect size (Cohen d=0.25) was found when comparing mean change over time on the primary outcome measure between the intervention and wait-list groups. In addition, a small effect size was found between pre- and postintervention assessments within the intervention group (Cohen d=0.38) as well as within the wait-list group (Cohen d=0.25).

CONCLUSIONS

The effect of MSC on perceived stress remains uncertain. As adherence was low in the intervention group, elements or features that facilitate adherence and engagement must be further developed before firmer conclusions regarding the effect of MSC can be made.

TRIAL REGISTRATION

ClinicalTrials.gov NCT03077568; https://clinicaltrials.gov/ct2/show/NCT03077568.

In Search of Digital Dopamine: How Apps Can Motivate Depressed Patients, a Review and Conceptual Analysis

INTRODUCTION

Depression is highly prevalent and causes considerable suffering and disease burden despite the existence of wide-ranging treatment options. Momentary assessment is a promising tool in the management of psychiatric disorders, and particularly depression. It allows for a real-time evaluation of symptoms and an earlier detection of relapse or treatment efficacy. Treating the motivational and hedonic aspects of depression is a key target reported in the literature, but it is time-consuming in terms of human resources. Digital Applications offer a major opportunity to indirectly regulate impaired motivational circuits through dopaminergic pathways.

OBJECTIVE

The main objective of this review was twofold: (1) propose a conceptual and critical review of the literature regarding the theoretical and technical principles of digital applications focused on motivation in depression, activating dopamine, and (2) suggest recommendations on the relevance of using these tools and their potential place in the treatment of depression.

MATERIAL AND METHODS

A search for words related to "dopamine", "depression", "smartphone apps", "digital phenotype" has been conducted on PubMed.

RESULTS

Ecological momentary interventions (EMIs) differ from traditional treatments by providing relevant, useful intervention strategies in the context of people's daily lives. EMIs triggered by ecological momentary assessment (EMA) are called "Smart-EMI". Smart-EMIs can mimic the "dopamine reward system" if the intervention is tailored for motivation or hedonic enhancement, and it has been shown that a simple reward (such as a digital badge) can increase motivation.

DISCUSSION

The various studies presented support the potential interest of digital health in effectively motivating depressed patients to adopt therapeutic activation behaviors. Finding effective ways to integrate EMIs with human-provided therapeutic support may ultimately yield the most efficient and effective intervention method. This approach could be a helpful tool to increase adherence and motivation.

CONCLUSION

Smartphone apps can motivate depressed patients by enhancing dopamine, offering the opportunity to enhance motivation and behavioral changes, although longer term studies are still needed.

Facilitator Contact, Discussion Boards, and Virtual Badges as Adherence Enhancements to a Web-Based, Self-guided, Positive Psychological Intervention for Depression: Randomized Controlled Trial

Background

Adherence to self-guided interventions tends to be very low, especially in people with depression. Prior studies have demonstrated that enhancements may increase adherence, but little is known about the efficacy of various enhancements in comparison to, or in combination with, one another.

Objective

The aim of our study is to test whether 3 enhancements—facilitator contact (FC), an online discussion board, and virtual badges (VB)—alone, or in combination, improve adherence to a self-guided, web-based intervention for depression. We also examined whether age, gender, race, ethnicity, comfort with technology, or baseline depression predicted adherence or moderated the effects that each enhancement had on adherence.

Methods

Participants were recruited through web-based sources and, after completing at least 4 out of 7 daily emotion reports, were sequentially assigned to 1 of 9 conditions—the intervention alone; the intervention plus 1, 2, or all 3 enhancements; or an emotion reporting control condition. The intervention was a positive psychological program consisting of 8 skills that specifically targeted positive emotions, and it was delivered over 5 weeks in a self-guided, web-based format. We operationalized adherence as the number of skills accessed.

Results

A total of 602 participants were enrolled in this study. Participants accessed, on average, 5.61 (SD 2.76) of 8 skills. The total number of enhancements participants received (0-3) did not predict the number of skills accessed. Participants who were assigned to the VB+FC condition accessed significantly more skills than those in the intervention only conditions. Furthermore, participants in arms that received the combination of both the VB and FC enhancements (VB+FC and VB+FC+online discussion board) accessed a greater number of skills relative to the number of skills accessed by participants who received either VB or FC without the other. Moderation analyses revealed that the receipt of VB (vs no VB) predicted higher adherence among participants with moderately severe depression at baseline.

Conclusions

The results suggested that the VB+FC combination significantly increased the number of skills accessed in a self-guided, web-based intervention for elevated depression. We have provided suggestions for refinements to these enhancements, which may further improve adherence.

Trial Registration

ClinicalTrials.gov NCT02861755; http://clinicaltrials.gov/ct2/show/NCT02861755

Conceptual Ambiguity Surrounding Gamification and Serious Games in Health Care: Literature Review and Development of Game-Based Intervention Reporting Guidelines (GAMING)

BACKGROUND

In health care, the use of game-based interventions to increase motivation, engagement, and overall sustainability of health behaviors is steadily becoming more common. The most prevalent types of game-based interventions in health care research are gamification and serious games. Various researchers have discussed substantial conceptual differences between these 2 concepts, supported by empirical studies showing differences in the effects on specific health behaviors. However, researchers also frequently report cases in which terms related to these 2 concepts are used ambiguously or even interchangeably. It remains unclear to what extent existing health care research explicitly distinguishes between gamification and serious games and whether it draws on existing conceptual considerations to do so.

OBJECTIVE

This study aims to address this lack of knowledge by capturing the current state of conceptualizations of gamification and serious games in health care research. Furthermore, we aim to provide tools for researchers to disambiguate the reporting of game-based interventions.

METHODS

We used a 2-step research approach. First, we conducted a systematic literature review of 206 studies, published in the Journal of Medical Internet Research and its sister journals, containing terms related to gamification, serious games, or both. We analyzed their conceptualizations of gamification and serious games, as well as the distinctions between the two concepts. Second, based on the literature review findings, we developed a set of guidelines for researchers reporting on game-based interventions and evaluated them with a group of 9 experts from the field.

RESULTS

Our results show that less than half of the concept mentions are accompanied by an explicit definition. To distinguish between the 2 concepts, we identified four common approaches: implicit distinction, synonymous use of terms, serious games as a type of gamified system, and distinction based on the full game dimension. Our Game-Based Intervention Reporting Guidelines (GAMING) consist of 25 items grouped into four topics: conceptual focus, contribution, mindfulness about related concepts, and individual concept definitions.

CONCLUSIONS

Conceptualizations of gamification and serious games in health care literature are strongly heterogeneous, leading to conceptual ambiguity. Following the GAMING can support authors in rigorous reporting on study results of game-based interventions.

Smartphone apps for depression and anxiety: a systematic review and meta-analysis of techniques to increase engagement

Meta-analyses have shown that digital mental health apps can be efficacious in reducing symptoms of depression and anxiety. However, real-world usage of apps is typically not sustained over time, and no studies systematically examine which features increase sustained engagement with apps or the relationship between engagement features and clinical efficacy. We conducted a systematic search of the literature to identify empirical studies that (1) investigate standalone apps for depression and/or anxiety in symptomatic participants and (2) report at least one measure of engagement. Features intended to increase engagement were categorized using the persuasive system design (PSD) framework and principles of behavioral economics. Twenty-five studies with 4159 participants were included in the analysis. PSD features were commonly used, whereas behavioral economics techniques were not. Smartphone apps were efficacious in treating symptoms of anxiety and depression in randomized controlled trials, with overall small-to-medium effects (g = 0.2888, SE = 0.0999, z(15) = 2.89, p = 0.0119, Q(df = 14) = 41.93, p < 0.0001, I2 = 66.6%), and apps that employed a greater number of engagement features as compared to the control condition had larger effect sizes (β = 0.0450, SE = 0.0164, t(15) = 2.7344, p = 0.0161). We observed an unexpected negative association between PSD features and engagement, as measured by completion rate (β = -0.0293, SE = 0.0121, t(17) = 02.4142, p = 0.0281). Overall, PSD features show promise for augmenting app efficacy, though engagement, as reflected in study completion, may not be the primary factor driving this association. The results suggest that expanding the use of PSD features in mental health apps may increase clinical benefits and that other techniques, such as those informed by behavioral economics, are employed infrequently.

Engaging youth in mHealth: what works and how can we be sure?

BACKGROUND

Youth participating in mobile health (mHealth) intervention trials often engage with the technologies [e.g., applications (app) or mobile-optimized websites] only partially, often prematurely discontinuing use altogether. Limited engagement can impact the interventions effect on behavior change and compromise researchers' ability to test and estimate the true efficacy of their interventions. While mHealth interventions have been shown to be feasible and acceptable to youth, across diverse health conditions, strategies to increase engagement have been less well studied. Specifically, within HIV prevention and care mHealth interventions, there is not consensus as to which components represent the "key ingredients" to support maximal engagement of youth. Further, successful intervention evaluation requires the ability to systematically track users' engagement with intervention components (i.e., paradata) to evaluate its effects on behavior change.

METHODS

As part of the Adolescent Medicine Trials Network UNC/Emory Center for Innovative Technology (iTech) portfolio of HIV/AIDS Interventions, we present diverse strategies used across five mHealth protocols seeking to promote youth engagement, track and measure engagement through paradata, and incorporate these components into mHealth intervention evaluations.

RESULTS

We describe the importance of defining and measuring engagement using case studies from iTech to illustrate how different research teams select mHealth features to promote youth engagement over time, taking into account features embedded in the technology design, key mechanisms of change and trial outcomes (e.g., HIV testing, pre-exposure prophylaxis uptake and adherence, HIV treatment adherence). Finally, we discuss how the research teams plan to evaluate engagement's role on their intervention's outcomes.

CONCLUSIONS

Based on this synthesis, we discuss strategies to enhance mHealth engagement during intervention development and design, ensure its monitoring and reporting throughout the trial, and evaluate its impact on trial outcomes.

Psychometric Evaluation of the TWente Engagement with Ehealth Technologies Scale (TWEETS): Evaluation Study

Background

Engagement emerges as a predictor for the effectiveness of digital health interventions. However, a shared understanding of engagement is missing. Therefore, a new scale has been developed that proposes a clear definition and creates a tool to measure it. The TWente Engagement with Ehealth Technologies Scale (TWEETS) is based on a systematic review and interviews with engaged health app users. It defines engagement as a combination of behavior, cognition, and affect.

Objective

This paper aims to evaluate the psychometric properties of the TWEETS. In addition, a comparison is made with the experiential part of the Digital Behavior Change Intervention Engagement Scale (DBCI-ES-Ex), a scale that showed some issues in previous psychometric analyses.

Methods

In this study, 288 participants were asked to use any step counter app on their smartphones for 2 weeks. They completed online questionnaires at 4 time points: T0=baseline, T1=after 1 day, T2=1 week, and T3=2 weeks. At T0, demographics and personality (conscientiousness and intellect/imagination) were assessed; at T1-T3, engagement, involvement, enjoyment, subjective usage, and perceived behavior change were included as measures that are theoretically related to our definition of engagement. Analyses focused on internal consistency, reliability, and the convergent, divergent, and predictive validity of both engagement scales. Convergent validity was assessed by correlating the engagement scales with involvement, enjoyment, and subjective usage; divergent validity was assessed by correlating the engagement scales with personality; and predictive validity was assessed by regression analyses using engagement to predict perceived behavior change at later time points.

Results

The Cronbach alpha values of the TWEETS were .86, .86, and .87 on T1, T2, and T3, respectively. Exploratory factor analyses indicated that a 1-factor structure best fits the data. The TWEETS is moderately to strongly correlated with involvement and enjoyment (theoretically related to cognitive and affective engagement, respectively; P<.001). Correlations between the TWEETS and frequency of use were nonsignificant or small, and differences between adherers and nonadherers on the TWEETS were significant (P<.001). Correlations between personality and the TWEETS were nonsignificant. The TWEETS at T1 was predictive of perceived behavior change at T3, with an explained variance of 16%. The psychometric properties of the TWEETS and the DBCI-ES-Ex seemed comparable in some aspects (eg, internal consistency), and in other aspects, the TWEETS seemed somewhat superior (divergent and predictive validity).

Conclusions

The TWEETS performs quite well as an engagement measure with high internal consistency, reasonable test-retest reliability and convergent validity, good divergent validity, and reasonable predictive validity. As the psychometric quality of a scale is a reflection of how closely a scale matches the conceptualization of a concept, this paper is also an attempt to conceptualize and define engagement as a unique concept, providing a first step toward an acceptable standard of defining and measuring engagement.

Feasibility of Self-Monitoring Rheumatoid Arthritis With a Smartphone App: Results of Two Mixed-Methods Pilot Studies

Background

Several mobile apps that monitor symptoms of rheumatoid arthritis (RA) exist, but a recent systematic review indicated that high-quality apps are lacking. When patients self-monitor their own disease with patient-reported outcomes (PROs) and self-initiate care at the right moment, it may be possible to reduce the frequency of their clinic visits, which would reduce health care burden and costs. We developed an app, that is, the MijnReuma Reade app, for this purpose and performed 2 pilot tests with weekly self-monitoring.

Objective

The primary objective of this study was to design, develop, and evaluate the usability, satisfaction, and usage of the MijnReuma Reade app—an app that allows patients with RA to monitor their own disease. The secondary objective was to review the patients’ perspectives on app usage and its intended purpose.

Methods

This app was designed in collaboration with patients with RA, rheumatologists, and information technology experts. Two 1-month pilot studies were performed, after which satisfaction (0-10 scale), usability (system usability scale, 0-100), and usage (proportion of completed questionnaires) of this app were assessed. After the second pilot study, semistructured interviews were performed to determine patients’ perspectives and the promoters and barriers of app usage.

Results

In the first and second pilot study, 42 and 27 patients were included, respectively. Overall, the patients were satisfied (medians, 8 and 7) and found the app usable (mean system usability scores, 76 and 71) in pilot studies 1 and 2, respectively. App usage declined over time in both the pilot studies; 61% (17/28) and 37% (10/27) of the patients who disclosed their usage statistics completed the final weekly questionnaire in pilot study 1 and pilot study 2, respectively. Approximately 81% (25/31) of the patients indicated they would like to skip hospital visits if the self-monitored disease activity is low. In the semistructured interviews, technical problems, internal resistance (respondent fatigue, the app reminded them of their disease), and a lack of symptoms were identified as barriers for usage. Patients reported that “experiencing more grip on their disease” and “improved communication with their physician” were promoters for usage. Patients reported that pain positively mediated usage, that is, more pain promoted and less pain discouraged app usage.

Conclusions

This study illustrates the feasibility of the MijnReuma Reade app that enables self-monitoring of the disease activity in patients with RA with the overarching aim to allocate clinical consultations according to need. Satisfaction with the app and usability of the app were found to be high; however, app usage declined over time. Patients acknowledged the potential of the app to self-monitor their own disease and would like to be able to skip clinic visits if the monitored disease activity is low. To evaluate this strategy, a randomized controlled trial is underway.

Gamification as an approach to improve resilience and reduce attrition in mobile mental health interventions: A randomized controlled trial

Forty percent of all general-practitioner appointments are related to mental illness, although less than 35% of individuals have access to therapy and psychological care, indicating a pressing need for accessible and affordable therapy tools. The ubiquity of smartphones offers a delivery platform for such tools. Previous research suggests that gamification-turning intervention content into a game format-could increase engagement with prevention and early-stage mobile interventions. This study aimed to explore the effects of a gamified mobile mental health intervention on improvements in resilience, in comparison with active and inactive control conditions. Differences between conditions on changes in personal growth, anxiety and psychological wellbeing, as well as differences in attrition rates, were also assessed. The eQuoo app was developed and published on all leading mobile platforms. The app educates users about psychological concepts including emotional bids, generalization, and reciprocity through psychoeducation, storytelling, and gamification. In total, 358 participants completed in a 5-week, 3-armed (eQuoo, "treatment as usual" cognitive behavioral therapy journal app, no-intervention waitlist) randomized controlled trial. Relevant scales were administered to all participants on days 1, 17, and 35. Repeated-measures ANOVA revealed statistically significant increases in resilience in the test group compared with both control groups over 5 weeks. The app also significantly increased personal growth, positive relations with others, and anxiety. With 90% adherence, eQuoo retained 21% more participants than the control or waitlist groups. Intervention delivered via eQuoo significantly raised mental well-being and decreased self-reported anxiety while enhancing adherence in comparison with the control conditions. Mobile apps using gamification can be a valuable and effective platform for well-being and mental health interventions and may enhance motivation and reduce attrition. Future research should measure eQuoo's effect on anxiety with a more sensitive tool and examine the impact of eQuoo on a clinical population.

The Concept and Components of Engagement in Different Domains Applied to eHealth: A Systematic Scoping Review

Within the context of eHealth interventions, a shared understanding of what constitutes engagement in and with eHealth technologies is missing. A clearer understanding of engagement could provide a valuable starting point for guidelines relating to the design and development of eHealth technologies. Given the cross-disciplinary use of the term "engagement," investigating how engagement (and its components) is conceptualized in different domains could lead to determining common components that are deemed important for eHealth technological design. As such, the aim of this paper was 3-fold: (a) to investigate in which domains engagement features, (b) to determine what constitutes engagement in these different domains, and (c) to determine whether there are any common components that seem to be important. A comprehensive systematic scoping review of the existing literature was conducted in order to identify the domains in which engagement is used, to extract the associated definitions of engagement, and to identify the dimensionality or components thereof. A search of five bibliographic databases yielded 1,231 unique records. All titles, abstracts, and full texts were screened based on specific inclusion and exclusion criteria. This led to 69 articles being included for further analyses. The results showed that engagement is used in seven functional domains, categorized as follows: student (n = 18), customer (n = 12), health (n = 11), society (n = 10), work (n = 9), digital (n = 8), and transdisciplinary (n = 1) domains. It seems that some domains are more mature regarding their conceptualization and theorizing on engagement than others. Further, engagement was found to be predominantly conceptualized as a multidimensional construct with three common components (behavior, cognition, and affective) shared between domains. Although engagement is prolifically used in different disciplines, it is evident that little shared consensus as to its conceptualization within and between domains exists. Despite this, engagement is foremost seen as a state of being engaged in/with something, which is part of, but should not be confused with, the process of engagement. Behavior, cognition, and affect are important components of engagement and should be specified for each new context.

Mental Health Therapy Protocols and eHealth Design: Focus Group Study

Background

Electronic health (eHealth) programs are often based on protocols developed for the original face-to-face therapies. However, in practice, therapists and patients may not always follow the original therapy protocols. This form of personalization may also interfere with the intended implementation and effects of eHealth interventions if designers do not take these practices into account.

Objective

The aim of this explorative study was to gain insights into the personalization practices of therapists and patients using cognitive behavioral therapy, one of the most commonly applied types of psychotherapy, in a youth addiction care center as a case context.

Methods

Focus group discussions were conducted asking therapists and patients to estimate the extent to which a therapy protocol was followed and about the type and reasons for personalization of a given therapy protocol. A total of 7 focus group sessions were organized involving therapists and patients. We used a commonly applied protocol for cognitive behavioral therapy as a therapy protocol example in youth mental health care. The first focus group discussions aimed at assessing the extent to which patients (N=5) or therapists (N=6) adapted the protocol. The second focus group discussions aimed at estimating the extent to which the therapy protocol is applied and personalized based on findings from the first focus groups to gain further qualitative insight into the reasons for personalization with groups of therapists and patients together (N=7). Qualitative data were analyzed using thematic analysis.

Results

Therapists used the protocol as a “toolbox” comprising different therapy tools, and personalized the protocol to enhance the therapeutic alliance and based on their therapy-provision experiences. Therapists estimated that they strictly follow 48% of the protocol, adapt 30%, and replace 22% by other nonprotocol therapeutic components. Patients personalized their own therapy to conform the assignments to their daily lives and routines, and to reduce their levels of stress and worry. Patients estimated that 29% of the provided therapy had been strictly followed by the therapist, 48% had been adjusted, and 23% had been replaced by other nonprotocol therapeutic components.

Conclusions

A standard cognitive behavioral therapy protocol is not strictly and fully applied but is mainly personalized. Based on these results, the following recommendations for eHealth designers are proposed to enhance alignment of eHealth to therapeutic practice and implementation: (1) study and copy at least the applied parts of a protocol, (2) co-design eHealth with therapists and patients so they can allocate the components that should be open for user customization, and (3) investigate if components of the therapy protocol that are not applied should remain part of the eHealth applied. To best generate this information, we suggest that eHealth designers should collaborate with therapists, patients, protocol developers, and mental health care managers during the development process.

Serious Gaming During Multidisciplinary Rehabilitation for Patients With Chronic Pain or Fatigue Symptoms: Mixed Methods Design of a Realist Process Evaluation

BACKGROUND

Serious gaming could support patients in learning to cope with chronic pain or functional somatic syndromes and reduce symptom burdens.

OBJECTIVE

To realize this potential, insight is needed into how, why, for whom, and when it works in actual treatment circumstances.

METHODS

Following a realist approach, process evaluations were performed before, during, and after a two-armed, natural quasi-experiment (n=275). A group of patients with interfering chronic pain or fatigue symptoms received a short additional blended mindfulness-based serious gaming intervention during a multidisciplinary rehabilitation program. A control group only received the regular rehabilitation program. During two sessions before and one session after the experiment, expectations about serious gaming processes were discussed in focus groups with local care providers, implementers, and experts. Patients participated in a survey (n=114) and in semistructured interviews (n=10). The qualitative data were used to develop tentative expectations about aspects of serious gaming that, in certain patients and circumstances, trigger mechanisms of learning and health outcome change. Hypotheses about indicative quantitative data patterns for tentative expectations were formulated before inspecting, describing, and analyzing-with regression models-routinely collected clinical outcome data. An updated program theory was formulated after mixing the qualitative and quantitative results.

RESULTS

Qualitative data showed that a subset of patients perceived improvement of their self-awareness in moments of daily social interactions. These results were explained by patients, who played the serious game LAKA, as a "confrontation with yourself," which reflected self-discrepancies. Important characteristics of serious gaming in the study's context included innovation factors of relative advantage with experiential learning opportunity, compatibility with the treatment approach, and the limited flexibility in regard to patient preferences. Perceived patient factors included age and style of coping with stress or pain. Learning perceptions could also depend on care provider role-taking and the planning and facilitating (ie, local organization) of serious gaming introduction and feedback sessions in small groups of patients. Quantitative data showed very small average differences between the study groups in self-reported depression, pain, and fatigue changes (-.07<beta<-.17, all 95% CI upper bounds <0), which were mediated by small group differences in mindfulness (beta=.26, 95% CI .02-.51). Mindfulness changes were positively associated with patient involvement in serious gaming (n=114, beta=.36, P=.001). Acceptance of serious gaming was lower in older patients. Average health outcome changes went up to a medium size in patients that reported lower active coping with stress and lower pain coping before serious gaming. Mindfulness changes and gaming acceptance perceptions covaried with group structure and immediate feedback sessions after serious gaming.

CONCLUSIONS

This study developed transferable insight into how and why serious gaming can facilitate additional learning about coping in order to reduce burdens of chronic pain or fatigue symptoms in certain patients and in actual treatment circumstances. Future studies are needed to continue the development of this fallible theory. Such research will further support decisions about using, designing, allocating, and tailoring serious gaming to optimize important patient health benefits.

TRIAL REGISTRATION

Netherlands Trial Register NTR6020; https://www.trialregister.nl/trial/5754.

Towards Scalable, Integrative Assessment of Children's Self-Regulatory Capabilities: New Applications of Digital Technology

The assessment of self-regulation in children is of significant interest to researchers within education, clinical and developmental psychology, and clinical neuroscience, given its importance to adaptive functioning across a wide range of social, educational, interpersonal, educational and health domains. Because self-regulation is a complex, multidimensional construct, a range of assessment approaches have been developed to assess its various components including behavioural, cognitive and emotional domains. In recent years, digital technology has been increasingly used to enhance or supplement existing measurement approaches; however, developments have predominantly focused on translating traditional testing paradigms into digital formats. There is a need for more innovation in digital psychological assessments that harness modern mechanisms such as game-based design and interactivity. Such approaches have potential for the development of scalable, adaptable universal approaches to screening and assessment of children's self-regulatory capabilities, to facilitate early identification of difficulties in individuals and also guide planning and decision-making at a population level. We highlight a novel, innovative digital assessment tool for children called Rumble's Quest, a new measure of children's socio-emotional functioning that shows promise as an integrative assessment of well-being and self-regulation, and which incorporates both self-report and direct assessment of cognitive self-regulation. This tool is scalable, can be integrated into normal classroom activities, and forms part of a comprehensive prevention support system that can be used to guide stakeholders' decision-making regarding early intervention and support at the individual, classroom, school and community level. We finish by discussing other innovative possibilities for psychological assessment with children, using new and emerging technologies and assessment approaches.

Gamifying Breastfeeding for Fathers: Process Evaluation of the Milk Man Mobile App

BACKGROUND

Mobile technology offers unique opportunities to reach people with health promotion interventions. Breastfeeding is an important public health issue, and fathers are a key support. Milk Man is a father-focused breastfeeding app that sought to engage fathers with information and conversation about breastfeeding, with the goal to impact positively on breastfeeding duration.

OBJECTIVE

The study aimed to describe the process evaluation of the Milk Man app that was trialed in the Parent Infant Feeding Initiative randomized controlled trial.

METHODS

The app used an information library, gamification, push notifications, and social connectivity via a Web-based conversation forum, which included polls and conversation starters, to engage fathers with breastfeeding information. Fathers had access to the app from approximately 32 weeks of gestation to 6 months postpartum. Process evaluation data were collected from a self-completed questionnaire administered via a Web-based link sent to participants at 6 weeks postpartum, and app analytics data were collected directly from the app. Quantitative data from both sources and qualitative responses to open-ended questions were used to triangulate findings to investigate patterns of usage and the effectiveness of each app engagement strategy to motivate and engage users.

RESULTS

A total of 80.3% (586/730) of participants, who were randomized to receive the app, downloaded Milk Man. Push notifications and interest in what other fathers had posted in the forum were the 2 main motivators to app use. Fathers used the app most while their partners were still pregnant and in the weeks immediately after the birth of their baby. Perspectives on the gamification strategy were varied. However, at 6 weeks postpartum, approximately one-third of fathers still using the app said that the gamification elements were encouraging the app use. The ease of use of the app and the design were important elements that were rated positively. The conversation forum emerged as the hub of app activity; all but 1 of the most accessed library articles and external organization links had been prompted as part of a conversation starter. Fathers posted comments in the conversation forum 1126 times (average of 2.21 per user) and voted in polls 3096 times (average of 6 per user).

CONCLUSIONS

These results demonstrate that the Milk Man app was an acceptable source of breastfeeding information and support that fathers and fathers-to-be are prepared to use throughout the perinatal period. The app showed encouraging results with facilitating conversation between partners. The conversation forum was clearly central to the success of the app, and fathers provided suggestions for improvement. Gamification results were varied, yet it was a key motivator for some users. These results provide valuable insight into the acceptability of the engagement strategies, including motivations for use and user perspectives on the app.

TRIAL REGISTRATION

Australian New Zealand Clinical Trials Registry ACTRN12614000605695; https://www.anzctr.org.au /Trial/Registration/TrialReview.aspx?ACTRN=12614000605695.

Using Gamification and Social Incentives to Increase Physical Activity and Related Social Cognition among Undergraduate Students in Shanghai, China

Gamification and social incentives are promising strategies to increase the effectiveness of web-based physical activity (PA) interventions by improving engagement. In this study, we designed a PA intervention integrating gamification and social incentives based on the most popular social networking service in China, WeChat. A controlled trial involving 52 Chinese undergraduate students was implemented to evaluate the effectiveness of the intervention. Subjects in the intervention group received a 7-week intervention. PA behavior and related social cognitive variables according to the theory of planned behavior were measured at the baseline and after the intervention. Daily physical activity duration was measured during the intervention. The results showed that PA-related subjective norms, perceived behavior control, and intention, as well as self-reported vigorous physical activity and moderate physical activity in the intervention group, were increased after the intervention, compared with the control group (p <0.05). During the intervention, perceived daily physical activity duration in the intervention group was on the rise, while it declined in the control group (p <0.001). The findings indicate that WeChat-based intervention integrating gamification and social incentives could effectively increase subjectively measured PA and related social cognition among Chinese undergraduate students and that it is a promising way to ameliorate the problem of insufficient PA among youths.