Measuring Engagement with Mental Health and Behavior Change Interventions: an Integrative Review of Methods and Instruments

The conceptualisation and measurement of engagement in digital health

Digital tools are an increasingly important component of healthcare, but their potential impact is commonly limited by a lack of user engagement. Digital health evaluations of engagement are often restricted to system usage metrics, which cannot capture a full understanding of how and why users engage with an intervention. This study aimed to examine how theory-based, multifaceted measures of engagement with digital health interventions capture different components of engagement (affective, cognitive, behavioural, micro, and macro) and to consider areas that are unclear or missing in their measurement. We identified and compared two recently developed measures that met these criteria (the Digital Behaviour Change Intervention Engagement Scale and the TWente Engagement with Ehealth Technologies Scale). Despite having similar theoretical bases and being relatively strongly correlated, there are key differences in how these scales aim to capture engagement. We discuss the implications of our analysis for how affective, cognitive, and behavioural components of engagement can be conceptualised and whether there is value in distinguishing between them. We conclude with recommendations for the circumstances in which each scale may be most useful and for how future measure development could supplement existing scales.

Reflective Engagement With a Digital Physical Activity Intervention Among People Living With and Beyond Breast Cancer: Mixed Methods Study

BACKGROUND

People living with and beyond breast cancer can face internal barriers to physical activity (eg, fatigue and pain). Digital interventions that promote psychological acceptance and motivation may help this population navigate these barriers. The degree to which individuals (1) adhere to intervention protocols and (2) reflect on and internalize intervention content may predict intervention efficacy.

OBJECTIVE

The objective of this study was to characterize the nature of reflective processes brought about by an 8-week acceptance- and mindfulness-based physical activity intervention for insufficiently active survivors of breast cancer (n=75). Furthermore, we explored the potential utility of a metric of reflective processes for predicting study outcomes.

METHODS

Of the intervention's 8 weekly modules, 7 (88%) included an item that asked participants to reflect on what they found to be most useful. Two coders conducted directed content analysis on participants' written responses. They assessed each comment's depth of reflection using an existing framework (ranging from 0 to 4, with 0=simple description and 4=fundamental change with consideration of social and ethical issues). The coders identified themes within the various levels of reflection. We fit multiple linear regression models to evaluate whether participants' (1) intervention adherence (ie, number of modules completed) and (2) the mean level of the depth of reflection predicted study outcomes.

RESULTS

Participants were aged on average 57.2 (SD 11.2) years, mostly non-Hispanic White (58/75, 77%), and mostly overweight or obese (54/75, 72%). Of the 407 responses to the item prompting personal reflection, 70 (17.2%) were rated as reflection level 0 (ie, description), 247 (60.7%) were level 1 (ie, reflective description), 74 (18.2%) were level 2 (ie, dialogic reflection), 14 (3.4%) were level 3 (ie, transformative reflection), and 2 (0.5%) were level 4 (ie, critical reflection). Lower levels of reflection were characterized by the acquisition of knowledge or expressing intentions. Higher levels were characterized by personal insight, commentary on behavior change processes, and a change of perspective. Intervention adherence was associated with increases in self-reported weekly bouts of muscle-strengthening exercise (B=0.26, SE 0.12, 95% CI 0.02-0.50) and decreases in sleep disturbance (B=-1.04, SE 0.50, 95% CI -0.06 to -2.02). The mean level of reflection was associated with increases in psychological acceptance (B=3.42, SE 1.70, 95% CI 0.09-6.75) and motivation for physical activity (ie, integrated regulation: B=0.55, SE 0.25, 95% CI 0.06-1.04).

CONCLUSIONS

We identified a useful method for understanding the reflective processes that can occur during digital behavior change interventions serving people living with and beyond breast cancer. Intervention adherence and the depth of reflection each predicted changes in study outcomes. Deeper reflection on intervention content was associated with beneficial changes in the determinants of sustained behavior change. More research is needed to investigate the relations among digital behavior change intervention use, psychological processes, and intervention efficacy.

Positive mental health of Latin American university professors: A scientific framework for intervention and improvement

The post-pandemic stage covid-19 has revealed overloads, ambiguities, and conflicts of teachers in the performance of new roles in hybrid classrooms that demanded an urgent adaptation, this highlighted the need for priority attention to the mental health of teachers, however, there are still insufficient studies that transcend the diagnosis and are committed to establish proposals for improvement. OBJECTIVE: This study aims to establish a proposal for the promotion of positive mental health (PMH). METHODS: The study was deployed from a qualitative approach; using an ethnomethodological design that allowed studying how teachers create meanings and sense in their work context, an appreciative interview was conducted with an affirmative theme that allowed teachers to expose their experiences that were systematized and processed with ATLAS. ti software. The application of the interview was conducted online through a Google form, during the months of February and March 2023. Three hundred university professors who experienced the pandemic in universities in Brazil, Chile, Colombia, Ecuador, Mexico, and Peru participated, based on a convenience sampling. RESULTS: The results of the deductive phase confirmed Lluch's PMH theoretical framework; however, new nuances or variations have been identified, which must be considered in the complex and dynamic nature of each PMH factor. From there, the results of the inductive phase allowed revealing emerging concepts, that is, new categories that would have the function of improving the PMH factors, which is why they have been denominated: dynamizing nuclei. PMH dynamizing nuclei are adjustment to work environment, soft skills, work-family balance, self-motivation, self-efficacy, subjective well-being, proactive strategies, engagement, resilience. CONCLUSIONS: Finally, with the results of both phases, the creation of an integrated model was generated, which was evaluated by six experts in a round of feedback, who highlighted the relevance of the findings and offered recommendations that were considered in the study. The new integrated model has revealed an interesting association, since it not only legitimizes the PMH's dynamizing cores, but also informs on which specific factor of the PMH these cores have the greatest impact, which has a high guiding value for intervention and improvement based on focused strategies.

Macro-engagement in mHealth: Exploring user engagement beyond the screen

While digital technology holds great promise for health and well-being, some users feel sceptical about the time they spend online and how they use their mobile devices. This attitude could hamper uptake of digital health technologies and engagement with them. This study uses the concept of macro-engagement as a starting point to investigate how users of digital behaviour change interventions (DBCIs) engage with their behaviour change goals beyond the screens of their tools. Thirty semi-structured interviews were conducted with individuals who take part in behaviour change processes in different ways (i.e. mental health professionals, digital health experts and users of DBCIs). A qualitative analysis of their data through a grounded theory approach highlighted a wide array of offscreen behaviors and strategies that complement a behavior change process offscreen. Furthermore, implications for designing technology that encourages progressive non-reliance on DBCI usage are drawn out.

Time-varying model of engagement with digital self reporting: Evidence from smoking cessation longitudinal studies

Objective

Insufficient engagement is a critical barrier impacting the utility of digital interventions and mobile health assessments. As a result, engagement itself is increasingly becoming a target of studies and interventions. The purpose of this study is to investigate the dynamics of engagement in mobile health data collection by exploring whether, how, and why response to digital self-report prompts change over time in smoking cessation studies.

Method

Data from two ecological momentary assessment (EMA) studies of smoking cessation among diverse smokers attempting to quit (N = 573) with a total of 65,974 digital self-report prompts. We operationalize engagement with self-reporting in term of prompts delivered and prompt response to capture both broad and more granular engagement in self-reporting, respectively. The data were analyzed to describe trends in prompt delivered and prompt response over time. Time-varying effect modeling (TVEM) was employed to investigate the time-varying effects of response to previous prompt and the average response rate on the likelihood of current prompt response.

Results

Although prompt response rates were relatively stable over days in both studies, the proportion of participants with prompts delivered declined steadily over time in one of the studies, indicating that over time, fewer participants charged the device and kept it turned on (necessary to receive at least one prompt per day). Among those who did receive prompts, response rates were relatively stable. In both studies, there is a significant, positive and stable relationship between response to previous prompt and the likelihood of response to current prompt throughout all days of the study. The relationship between the average response rate prior to current prompt and the likelihood of responding to the current prompt was also positive, and increasing with time.

Conclusion

Our study highlights the importance of integrating various indicators to measure engagement in digital self-reporting. Both average response rate and response to previous prompt were highly predictive of response to the next prompt across days in the study. Dynamic patterns of engagement in digital self-reporting can inform the design of new strategies to promote and optimize engagement in digital interventions and mobile health studies.