Maximizing Participant Engagement, Participation, and Retention in Cohort Studies Using Digital Methods: Rapid Review to Inform the Next Generation of Very Large Birth Cohorts

Engaging Adolescents in Contemporary Longitudinal Health Research: Strategies for Promoting Participation and Retention

PURPOSE

High (and nonselective) recruitment and retention rates in longitudinal studies of adolescence are essential for illuminating health trajectories and determinants during this critical period. Knowledge of optimal recruitment and retention strategies must keep pace with emerging challenges and opportunities, such as the shifts towards digitally-based data collection.

METHODS

We used a narrative review approach to synthesize research on promising recruitment and retention strategies for optimizing engagement in the next generation of longitudinal adolescent health studies.

RESULTS

We identified a small number of well-evidenced strategies, emerging challenges and opportunities for recruitment and retention in contemporary studies, and key evidence gaps. Core recommendations include the use of well-evidenced strategies (e.g., incentivizing participation, reducing barriers and burden, and investing in building positive relationships with participants) and coproducing recruitment and retention strategies with adolescents and parents of adolescents.

DISCUSSION

More research is needed into successful recruitment/retention strategies for digital/remote data collection methods, but initial evidence suggests that adopting principles and adapting well-evidenced strategies from traditional longitudinal studies is promising.

Consideration of Sex, Gender, or Age on Outcomes of Digital Technologies for Treatment and Monitoring of Chronic Obstructive Pulmonary Disease: Overview of Systematic Reviews

BACKGROUND

Several systematic reviews have addressed digital technology use for treatment and monitoring of chronic obstructive pulmonary disease (COPD).

OBJECTIVE

This study aimed to assess if systematic reviews considered the effects of sex, gender, or age on the outcomes of digital technologies for treatment and monitoring of COPD through an overview of such systematic reviews. The objectives of this overview were to (1) describe the definitions of sex or gender used in reviews; (2) determine whether the consideration of sex, gender, or age was planned in reviews; (3) determine whether sex, gender, or age was reported in review results; (4) determine whether sex, gender, or age was incorporated in implications for clinical practice in reviews; and (5) create an evidence map for development of individualized clinical recommendations for COPD based on sex, gender, or age diversity.

METHODS

MEDLINE, the Cochrane Library, Epistemonikos, Web of Science, and the bibliographies of the included systematic reviews were searched to June 2022. Inclusion was based on the PICOS framework: (1) population (COPD), (2) intervention (any digital technology), (3) comparison (any), (4) outcome (any), and (5) study type (systematic review). Studies were independently selected by 2 authors based on title and abstract and full-text screening. Data were extracted by 1 author and checked by another author. Data items included systematic review characteristics; PICOS criteria; and variables related to sex, gender, or age. Systematic reviews were appraised using A Measurement Tool to Assess Systematic Reviews, version 2 (AMSTAR 2). Data were synthesized using descriptive statistics.

RESULTS

Of 1439 records, 30 systematic reviews published between 2010 and 2022 were included in this overview. The confidence in the results of 25 of the 30 (83%) reviews was critically low according to AMSTAR 2. The reviews focused on user outcomes that potentially depend on sex, gender, or age, such as efficacy or effectiveness (25/30, 83%) and acceptance, satisfaction, or adherence (3/30, 10%) to digital technologies for COPD. Reviews reported sex or gender (19/30 systematic reviews) or age (25/30 systematic reviews) among primary study characteristics. However, only 1 of 30 reviews included age in a subgroup analysis, and 3 of 30 reviews identified the effects of sex, gender, or age as evidence gaps.

CONCLUSIONS

This overview shows that the effects of sex, gender, or age were rarely considered in 30 systematic reviews of digital technologies for COPD treatment and monitoring. Furthermore, systematic reviews did not incorporate sex, gender, nor age in their implications for clinical practice. We recommend that future systematic reviews should (1) evaluate the effects of sex, gender, or age on the outcomes of digital technologies for treatment and monitoring of COPD and (2) better adhere to reporting guidelines to improve the confidence in review results.

TRIAL REGISTRATION

PROSPERO CRD42022322924; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=322924.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.2196/40538.

Increasing Engagement in the Electronic Framingham Heart Study: Factorial Randomized Controlled Trial

BACKGROUND

Smartphone apps and mobile health devices offer innovative ways to collect longitudinal cardiovascular data. Randomized evidence regarding effective strategies to maintain longitudinal engagement is limited.

OBJECTIVE

This study aimed to evaluate smartphone messaging interventions on remote transmission of blood pressure (BP) and heart rate (HR) data.

METHODS

We conducted a 2 × 2 × 2 factorial blinded randomized trial with randomization implemented centrally to ensure allocation concealment. We invited participants from the Electronic Framingham Heart Study (eFHS), an e-cohort embedded in the FHS, and asked participants to measure their BP (Withings digital cuff) weekly and wear their smartwatch daily. We assessed 3 weekly notification strategies to promote adherence: personalized versus standard; weekend versus weekday; and morning versus evening. Personalized notifications included the participant's name and were tailored to whether or not data from the prior week were transmitted to the research team. Intervention notification messages were delivered weekly automatically via the eFHS app. We assessed if participants transmitted at least one BP or HR measurement within 7 days of each notification after randomization. Outcomes were adherence to BP and HR transmission at 3 months (primary) and 6 months (secondary).

RESULTS

Of the 791 FHS participants, 655 (82.8%) were eligible and randomized (mean age 53, SD 9 years; 392/655, 59.8% women; 596/655, 91% White). For the personalized versus standard notifications, 38.9% (126/324) versus 28.8% (94/327) participants sent BP data at 3 months (difference=10.1%, 95% CI 2.9%-17.4%; P=.006), but no significant differences were observed for HR data transmission (212/324, 65.4% vs 209/327, 63.9%; P=.69). Personalized notifications were associated with increased BP and HR data transmission versus standard at 6 months (BP: 107/291, 36.8% vs 66/295, 22.4%; difference=14.4%, 95% CI 7.1- 21.7%; P<.001; HR: 186/281, 66.2% vs 158/281, 56.2%; difference=10%, 95% CI 2%-18%; P=.02). For BP and HR primary or secondary outcomes, there was no evidence of differences in data transmission for notifications sent on weekend versus weekday or morning versus evening.

CONCLUSIONS

Personalized notifications increased longitudinal adherence to BP and HR transmission from mobile and digital devices among eFHS participants. Our results suggest that personalized messaging is a powerful tool to promote adherence to mobile health systems in cardiovascular research.

TRIAL REGISTRATION

ClinicalTrials.gov NCT03516019; https://clinicaltrials.gov/ct2/show/NCT03516019.

Participant retention in a fully remote trial of digital psychotherapy: Comparison of incentive types

Numerous studies have found that long term retention is very low in remote clinical studies (&gt;4 weeks) and to date there is limited information on the best methods to ensure retention. The ability to retain participants in the completion of key assessments periods is critical to all clinical research, and to date little is known as to what methods are best to encourage participant retention. To study incentive-based retention methods we randomized 215 US adults (18+ years) who agreed to participate in a sequential, multiple assignment randomized trial to either high monetary incentive (HMI, $125 USD) and combined low monetary incentive ($75 USD) plus alternative incentive (LMAI). Participants were asked to complete daily and weekly surveys for a total of 12 weeks, which included a tailoring assessment around week 5 to determine who should be stepped up and rerandomized to one of two augmentation conditions. Key assessment points were weeks 5 and 12. There was no difference in participant retention at week 5 (tailoring event), with approximately 75% of the sample completing the week-5 survey. By week 10, the HMI condition retained approximately 70% of the sample, compared to 60% of the LMAI group. By week 12, all differences were attenuated. Differences in completed measures were not significant between groups. At the end of the study, participants were asked the impressions of the incentive condition they were assigned and asked for suggestions for improving engagement. There were no significant differences between conditions on ratings of the fairness of compensation, study satisfaction, or study burden, but study burden, intrinsic motivation and incentive fairness did influence participation. Men were also more likely to drop out of the study than women. Qualitative analysis from both groups found the following engagement suggestions: desire for feedback on survey responses and an interest in automated sharing of individual survey responses with study therapists to assist in treatment. Participants in the LMAI arm indicated that the alternative incentives were engaging and motivating. In sum, while we were able to increase engagement above what is typical for such study, more research is needed to truly improve long term retention in remote trials.

Evaluation of Digital Interventions for Physical Activity Promotion: Scoping Review

Background

Digital interventions are interventions supported by digital tools or technologies, such as mobile apps, wearables, or web-based software. Digital interventions in the context of public health are specifically designed to promote and improve health. Recent reviews have shown that many digital interventions target physical activity promotion; however, it is unclear how such digital interventions are evaluated.

Objective

We aimed to investigate evaluation strategies in the context of digital interventions for physical activity promotion using a scoping review of published reviews. We focused on the target (ie, user outcomes or tool performance), methods (ie, tool data or self-reported data), and theoretical frameworks of the evaluation strategies.

Methods

A protocol for this study was preregistered and published. From among 300 reviews published up to March 19, 2021 in Medline, PsycINFO, and CINAHL databases, 40 reviews (1 rapid, 9 scoping, and 30 systematic) were included in this scoping review. Two authors independently performed study selection and data coding. Consensus was reached by discussion. If applicable, data were coded quantitatively into predefined categories or qualitatively using definitions or author statements from the included reviews. Data were analyzed using either descriptive statistics, for quantitative data (relative frequencies out of all studies), or narrative synthesis focusing on common themes, for qualitative data.

Results

Most reviews that were included in our scoping review were published in the period from 2019 to 2021 and originated from Europe or Australia. Most primary studies cited in the reviews included adult populations in clinical or nonclinical settings, and focused on mobile apps or wearables for physical activity promotion. The evaluation target was a user outcome (efficacy, acceptability, usability, feasibility, or engagement) in 38 of the 40 reviews or tool performance in 24 of the 40 reviews. Evaluation methods relied upon objective tool data (in 35/40 reviews) or other data from self-reports or assessments (in 28/40 reviews). Evaluation frameworks based on behavior change theory, including goal setting, self-monitoring, feedback on behavior, and educational or motivational content, were mentioned in 22 out of 40 reviews. Behavior change theory was included in the development phases of digital interventions according to the findings of 20 out of 22 reviews.

Conclusions

The evaluation of digital interventions is a high priority according to the reviews included in this scoping review. Evaluations of digital interventions, including mobile apps or wearables for physical activity promotion, typically target user outcomes and rely upon objective tool data. Behavior change theory may provide useful guidance not only for development of digital interventions but also for the evaluation of user outcomes in the context of physical activity promotion. Future research should investigate factors that could improve the efficacy of digital interventions and the standardization of terminology and reporting in this field.

International Registered Report Identifier (IRRID)

RR2-10.2196/35332