Unconditional and conditional monetary incentives to increase response to mailed questionnaires: A randomized controlled study within a trial (SWAT)

Recruitment and retention of adolescents for an ecological momentary assessment measurement burst mental health study: The MHIM engagement strategy

INTRODUCTION

Recruitment and long-term retention of adolescent participants in longitudinal research are challenging and may be especially so in studies involving remote measurement and biosampling components. The ability to effectively recruit and retain participants can be supported by the use of specific evidence-based engagement strategies that are built in from the earliest stages.

METHODS

Informed by a review of the evidence on effective engagement strategies and consultations with adolescents (via two Young Person Advisory Groups [YPAGs]; ages 11-13 and 14-17), the current protocol describes the planned participant engagement strategy for the Mental Health in the Moment Study: a multimodal measurement burst study of adolescent mental health across ages 11-19.

RESULTS

The protocol incorporates engagement strategies in four key domains: consultations/co-design with the target population, incentives, relationship-building and burden/barrier reduction. In addition to describing general engagement strategies in longitudinal studies, we also discuss specific concerns regarding engagement in data collection methods such as biosampling and ecological momentary assessment where a paucity of evidence exists.

CONCLUSION

Engagement strategies for adolescent mental health studies should be based on existing evidence and consultations with adolescents. We present our approach in developing the planned engagement strategies and also discuss limitations and future directions in engaging adolescents in longitudinal research.

PATIENT OR PUBLIC CONTRIBUTION

The study design for this project places a strong emphasis on the active engagement of adolescents throughout its development. Specifically, the feedback and suggestions provided by the YPAGs have been instrumental in refining our strategies for maximising the recruitment and retention of participants.

Optimising recruitment to a lung cancer screening trial: A comparison of general practitioner and community-based recruitment

OBJECTIVES

Pre-trial focus groups of the Early detection of Cancer of the Lung Scotland (ECLS) trial indicated that those at high risk of lung cancer are more likely to engage with community-based recruitment methods. The current study aimed to understand if general practitioner (GP) and community-based recruitment might attract different groups of people, and to quantitatively explore the demographic and psychosocial differences between people responding to GP or community-based recruitment.

DESIGN

Secondary data analysis of ECLS trial baseline data.

METHODS

Adults (n = 11,164) aged 50 to 75 years completed a baseline questionnaire as part of their participation in the ECLS trial. The questionnaire assessed smoking behaviour, health state, health anxiety and illness perception. Alongside demographic characteristics, how participants were made aware of the study/participant recruitment method (GP recruitment/community recruitment) was also obtained via trial records.

RESULTS

The likelihood of being recruited via community-based methods increased as deprivation level decreased. Those recruited via the community had higher levels of perceived personal control of developing lung cancer and were more likely to understand their own risk of developing lung cancer, compared to those who were recruited to the trial via their GP. Health state and health anxiety did not predict recruitment methods in multivariable analysis.

CONCLUSIONS

Community and opportunistic screening invitations were associated with uptake in people from less-deprived backgrounds, and therefore might not be the optimal method to reach those at high risk of lung cancer and living in more deprived areas.

Engaging Adolescents in Contemporary Longitudinal Health Research: Strategies for Promoting Participation and Retention

PURPOSE

High (and nonselective) recruitment and retention rates in longitudinal studies of adolescence are essential for illuminating health trajectories and determinants during this critical period. Knowledge of optimal recruitment and retention strategies must keep pace with emerging challenges and opportunities, such as the shifts towards digitally-based data collection.

METHODS

We used a narrative review approach to synthesize research on promising recruitment and retention strategies for optimizing engagement in the next generation of longitudinal adolescent health studies.

RESULTS

We identified a small number of well-evidenced strategies, emerging challenges and opportunities for recruitment and retention in contemporary studies, and key evidence gaps. Core recommendations include the use of well-evidenced strategies (e.g., incentivizing participation, reducing barriers and burden, and investing in building positive relationships with participants) and coproducing recruitment and retention strategies with adolescents and parents of adolescents.

DISCUSSION

More research is needed into successful recruitment/retention strategies for digital/remote data collection methods, but initial evidence suggests that adopting principles and adapting well-evidenced strategies from traditional longitudinal studies is promising.

Methods to increase response to postal and electronic questionnaires

BACKGROUND

Self-administered questionnaires are widely used to collect data in epidemiological research, but non-response reduces the effective sample size and can introduce bias. Finding ways to increase response to postal and electronic questionnaires would improve the quality of epidemiological research.

OBJECTIVES

To identify effective strategies to increase response to postal and electronic questionnaires.

SEARCH METHODS

We searched 14 electronic databases up to December 2021 and manually searched the reference lists of relevant trials and reviews. We contacted the authors of all trials or reviews to ask about unpublished trials; where necessary, we also contacted authors to confirm the methods of allocation used and to clarify results presented.

SELECTION CRITERIA

Randomised trials of methods to increase response to postal or electronic questionnaires. We assessed the eligibility of each trial using pre-defined criteria.

DATA COLLECTION AND ANALYSIS

We extracted data on the trial participants, the intervention, the number randomised to intervention and comparison groups and allocation concealment. For each strategy, we estimated pooled odds ratios (OR) and 95% confidence intervals (CI) in a random-effects model. We assessed evidence for selection bias using Egger's weighted regression method and Begg's rank correlation test and funnel plot. We assessed heterogeneity amongst trial odds ratios using a Chi2 test and quantified the degree of inconsistency between trial results using the I2 statistic.

MAIN RESULTS

Postal We found 670 eligible trials that evaluated over 100 different strategies of increasing response to postal questionnaires. We found substantial heterogeneity amongst trial results in half of the strategies. The odds of response almost doubled when: using monetary incentives (odds ratio (OR) 1.86; 95% confidence interval (CI) 1.73 to 1.99; heterogeneity I2 = 85%); using a telephone reminder (OR 1.96; 95% CI 1.03 to 3.74); and when clinical outcome questions were placed last (OR 2.05; 95% CI 1.00 to 4.24). The odds of response increased by about half when: using a shorter questionnaire (OR 1.58; 95% CI 1.40 to 1.78); contacting participants before sending questionnaires (OR 1.36; 95% CI 1.23 to 1.51; I2 = 87%); incentives were given with questionnaires (i.e. unconditional) rather than when given only after participants had returned their questionnaire (i.e. conditional on response) (OR 1.53; 95% CI 1.35 to 1.74); using personalised SMS reminders (OR 1.53; 95% CI 0.97 to 2.42); using a special (recorded) delivery service (OR 1.68; 95% CI 1.36 to 2.08; I2 = 87%); using electronic reminders (OR 1.60; 95% CI 1.10 to 2.33); using intensive follow-up (OR 1.69; 95% CI 0.93 to 3.06); using a more interesting/salient questionnaire (OR 1.73; 95% CI 1.12 to 2.66); and when mentioning an obligation to respond (OR 1.61; 95% CI 1.16 to 2.22). The odds of response also increased with: non-monetary incentives (OR 1.16; 95% CI 1.11 to 1.21; I2 = 80%); a larger monetary incentive (OR 1.24; 95% CI 1.15 to 1.33); a larger non-monetary incentive (OR 1.15; 95% CI 1.00 to 1.33); when a pen was included (OR 1.44; 95% CI 1.38 to 1.50); using personalised materials (OR 1.15; 95% CI 1.09 to 1.21; I2 = 57%); using a single-sided rather than a double-sided questionnaire (OR 1.13; 95% CI 1.02 to 1.25); using stamped return envelopes rather than franked return envelopes (OR 1.23; 95% CI 1.13 to 1.33; I2 = 69%), assuring confidentiality (OR 1.33; 95% CI 1.24 to 1.42); using first-class outward mailing (OR 1.11; 95% CI 1.02 to 1.21); and when questionnaires originated from a university (OR 1.32; 95% CI 1.13 to 1.54). The odds of response were reduced when the questionnaire included questions of a sensitive nature (OR 0.94; 95% CI 0.88 to 1.00). Electronic We found 88 eligible trials that evaluated over 30 different ways of increasing response to electronic questionnaires. We found substantial heterogeneity amongst trial results in half of the strategies. The odds of response tripled when: using a brief letter rather than a detailed letter (OR 3.26; 95% CI 1.79 to 5.94); and when a picture was included in an email (OR 3.05; 95% CI 1.84 to 5.06; I2 = 19%). The odds of response almost doubled when: using monetary incentives (OR 1.88; 95% CI 1.31 to 2.71; I2 = 79%); and using a more interesting topic (OR 1.85; 95% CI 1.52 to 2.26). The odds of response increased by half when: using non-monetary incentives (OR 1.60; 95% CI 1.25 to 2.05); using shorter e-questionnaires (OR 1.51; 95% CI 1.06 to 2.16; I2 = 94%); and using a more interesting e-questionnaire (OR 1.85; 95% CI 1.52 to 2.26). The odds of response increased by a third when: offering survey results as an incentive (OR 1.36; 95% CI 1.16 to 1.59); using a white background (OR 1.31; 95% CI 1.10 to 1.56); and when stressing the benefits to society of response (OR 1.38; 95% CI 1.07 to 1.78; I2 = 41%). The odds of response also increased with: personalised e-questionnaires (OR 1.24; 95% CI 1.17 to 1.32; I2 = 41%); using a simple header (OR 1.23; 95% CI 1.03 to 1.48); giving a deadline (OR 1.18; 95% CI 1.03 to 1.34); and by giving a longer time estimate for completion (OR 1.25; 95% CI 0.96 to 1.64). The odds of response were reduced when: "Survey" was mentioned in the e-mail subject (OR 0.81; 95% CI 0.67 to 0.97); when the email or the e-questionnaire was from a male investigator, or it included a male signature (OR 0.55; 95% CI 0.38 to 0.80); and by using university sponsorship (OR 0.84; 95%CI 0.69 to 1.01). The odds of response using a postal questionnaire were over twice those using an e-questionnaire (OR 2.33; 95% CI 2.25 to 2.42; I2 = 98%). Response also increased when: providing a choice of response mode (electronic or postal) rather than electronic only (OR 1.76 95% CI 1.67 to 1.85; I2 = 97%); and when administering the e-questionnaire by computer rather than by smartphone (OR 1.62 95% CI 1.36 to 1.94).

AUTHORS' CONCLUSIONS

Researchers using postal and electronic questionnaires can increase response using the strategies shown to be effective in this Cochrane review.

What Motivates Physicians to Address Caregiver Needs? The Role of Experiential Similarity

Despite the significant stress of family caregiving, caregivers' needs and risks are often overlooked in healthcare settings. This study examined the factors associated with primary care physicians' perceived responsibility to identify and address caregiver needs and risks. Using a national random sample of U.S. primary care physicians (N = 106), multivariable logistic regression analyses were conducted to examine associations of physicians' perceived responsibility to assess caregivers' needs with experiential similarity (personal experience with caregiving), structural similarity (being older and female), and secondary exposure variables (time seeing older adults in the outpatient setting). Most (76.5%) physicians felt responsible for identifying caregivers' needs and risks. In multivariable models, physicians who had personal experience with caregiving were four times more likely than those without it to feel responsible for identifying caregivers' needs and risks and assessing caregivers' mental health concerns. Thus, physicians may benefit from educational interventions that immerse them in caregivers' lived experiences.

Postal Delivery of Sleep Monitoring Devices: Research Implications

The purpose of this article is to describe the protocol development, feasibility, and lessons learned in the postal mail delivery of sleep monitoring devices to study participants. The original study protocol included four in-person visits with distribution of a sleep monitoring device (Actiwatch) and return of the Actiwatch via the postal service in a self-addressed, stamped envelope. The COVID-19 pandemic limited in-person research contact thus requiring a remote study protocol for application and return of the Actiwatches using postal delivery. While there were postal delivery and return challenges, the overall return rate of 94.4% confirmed remote protocol feasibility. Key lessons learned were: consistent and frequent communication via telephone calls and/or text; confirming required postage; and use of package tracking labels. All these strategies contributed to successful postal delivery/return and concomitantly decreased the potential loss of data and valuable research equipment.

Impact of different unconditional monetary incentives on survey response rates in men with prostate cancer: a 2-arm randomised trial

Background

Men are often viewed as a difficult group to recruit for psychological research, including in psycho-oncology. Whilst research has demonstrated the effectiveness of small monetary incentives for encouraging research participation, little research has examined different large unconditional incentive amounts. Larger unconditional incentives may result in increased participation of men in psychological research. This randomised study within a case–control trial of men diagnosed with early-stage prostate cancer aimed to investigate whether (a) response rates to a 30-min questionnaire completed via mail, online, or phone would vary with different unconditional incentive amounts, and (b) demographics would vary in those who responded within the different incentive groups.

Methods

We conducted this randomised study within a case–control cross-sectional study aiming to identify the social-ecological factors influencing treatment discontinuation in prostate cancer patients. A total of 238 participants from the cross-sectional study were randomised to receive one of two unconditional incentives (n = 121 received AUD$10, n = 117 received AUD$20) with the study materials (consent form and survey).

Results

Overall, 113 (47%) responded; n = 61/121 (50.4%) in the AUD$10 group, and n = 52/117 (44.4%) in the AUD$20 group. No evidence of a difference was found in response rates by incentive group (odds ratio 1.27, 95% CI = 0.76–2.12, p = 0.36). Additionally, there were no evident differences in the demographics of the responders vs. non-responders within each incentive group (all p > 0.05).

Conclusions

Unlike previous research, we were unable to show that higher monetary incentives were more effective for increasing response rates. An AUD$20 unconditional incentive may be no more effective than a lesser amount for encouraging prostate cancer survivors to participate in research involving long questionnaires. Future research should consider the cost-benefits of providing large unconditional incentives, as non-responses will result in lost resources perhaps better utilised in other engagement strategies.

Incentive delivery timing and follow-up survey completion in a prospective cohort study of injured children: a randomized experiment comparing prepaid and postpaid incentives

Background

Retaining participants over time is a frequent challenge in research studies evaluating long-term health outcomes. This study’s objective was to compare the impact of prepaid and postpaid incentives on response to a six-month follow-up survey.

Methods

We conducted an experiment to compare response between participants randomized to receive either prepaid or postpaid cash card incentives within a multisite study of children under 15 years in age who were hospitalized for a serious, severe, or critical injury. Participants were parents or guardians of enrolled children. The primary outcome was survey response. We also examined whether demographic characteristics were associated with response and if incentive timing influenced the relationship between demographic characteristics and response. We evaluated whether incentive timing was associated with the number of calls needed for contact.

Results

The study enrolled 427 children, and parents of 420 children were included in this analysis. Follow-up survey response did not differ according to the assigned treatment arm, with the percentage of parents responding to the survey being 68.1% for the prepaid incentive and 66.7% with the postpaid incentive. Likelihood of response varied by demographics. Spanish-speaking parents and parents with lower income and lower educational attainment were less likely to respond. Parents of Hispanic/Latino children and children with Medicaid insurance were also less likely to respond. We found no relationship between the assigned incentive treatment and the demographics of respondents compared to non-respondents.

Conclusions

Prepaid and postpaid incentives can obtain similar participation in longitudinal pediatric critical care outcomes research. Incentives alone do not ensure retention of all demographic subgroups. Strategies for improving representation of hard-to-reach populations are needed to address health disparities and ensure the generalizability of studies using these results.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-021-01421-8.

An Empirical Study of How Household Energy Consumption Is Affected by Co-Owning Different Technological Means to Produce Renewable Energy and the Production Purpose

The transition from fossil fuel-based to renewable energy sources is one of the main economic and social challenges of the early 21st century. Due to the volatile character of wind and solar power production, matching supply and demand is essential for this transition to be successful. In this context, the willingness of private consumers to use energy flexibly has gained growing attention. Research indicates that a viable driver to motivate consumers to be demand flexible is to make them (co-)owners of renewable energy production facilities. However, existing research has only analyzed this question from an aggregated perspective. This article analyses whether behavioral changes triggered by (co-)ownership in renewables differ according to the type of installation; be it solar, wind, or bioenergy. In addition, the prosumption options self-consumption/self-consumption and sale/sale are considered. To do so, we collected 2074 completed questionnaires on energy consumption that entered an econometric model using propensity score matching to control for estimation biases. We find significant differences in the willingness to consume electricity in a flexible manner for (co-)owners of solar installations. However, only the usage of household appliances proves to be statistically significant (p-value = 0.04). Furthermore, the results show that within the group of (co-)owners of solar installation, the choice between self-consumption and sale of the produced energy has a significant effect on the inclination to become demand flexible (p-value ≤ 0.001; p-value = 0.003).

Impact of mobile phone delivered reminders and unconditional incentives on measles-containing vaccine timeliness and coverage: a randomised controlled trial in western Kenya

Introduction

Short message service (SMS) reminders coupled with a small monetary incentive conditioned on prompt vaccination have been shown to improve first-dose measles-containing vaccine (MCV1) uptake. We assessed whether SMS reminders and unconditional monetary incentives—more amenable to programmatic implementation—can improve MCV1 uptake in Kenya.

Methods

Caregivers of eligible infants aged 6–8 months were enrolled into an individually randomised controlled trial and assigned to receive either: no intervention (control), two SMS reminders (SMS) sent 3 days, and 1 day before the scheduled MCV1 date, or SMS reminders coupled with a Kenya Shilling (KES) 150 incentive (SMS +150 KES) sent 3 days before the scheduled MCV1 date. Study staff conducted a household follow-up visit at age 12 months to ascertain vaccination status. Log-binomial regression was used to estimate the relative and absolute difference in MCV1 timely coverage (by age 10 months), the primary outcome.

Results

Between 6 December 2016 and 31 March 2017, 179 infants were enrolled into each of the three study arms. Follow-up visits were completed between 19 April 2017 and 8 October 2017 for control (n=170), SMS (n=157) and SMS + 150 KES (n=158) children. MCV1 timely coverage was 68% among control arm infants compared with 78% in each intervention arm. This represented a non-statistically significant increase in the SMS arm (adjusted relative risk 1.13; 95% CI 0.99 to 1.30; p=0.070; adjusted risk difference 9.2%; 95% CI: −0.6 to 19.0%; p=0.066), but a statistically significant increase in the SMS + 150 KES arm (1.16; 95% CI 1.01 to 1.32; p=0.035; 10.6%; 95% CI 0.8 to 20.3%; p=0.034).

Conclusion

These findings suggest that the effect of SMS reminders coupled with a small unconditional monetary incentive on MCV1 uptake is comparable to that of SMS reminders alone, limiting their utility. Further studies in the absence of unexpected supply-side constraints are needed.

Trial registration number

NCT02904642

Strategies to improve retention in randomised trials

BACKGROUND

Poor retention of participants in randomised trials can lead to missing outcome data which can introduce bias and reduce study power, affecting the generalisability, validity and reliability of results. Many strategies are used to improve retention but few have been formally evaluated.

OBJECTIVES

To quantify the effect of strategies to improve retention of participants in randomised trials and to investigate if the effect varied by trial setting.

SEARCH METHODS

We searched the Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE, Scopus, PsycINFO, CINAHL, Web of Science Core Collection (SCI-expanded, SSCI, CPSI-S, CPCI-SSH and ESCI) either directly with a specified search strategy or indirectly through the ORRCA database. We also searched the SWAT repository to identify ongoing or recently completed retention trials. We did our most recent searches in January 2020.

SELECTION CRITERIA

We included eligible randomised or quasi-randomised trials of evaluations of strategies to increase retention that were embedded in 'host' randomised trials from all disease areas and healthcare settings. We excluded studies aiming to increase treatment compliance.

DATA COLLECTION AND ANALYSIS

We extracted data on: the retention strategy being evaluated; location of study; host trial setting; method of randomisation; numbers and proportions in each intervention and comparator group. We used a risk difference (RD) and 95% confidence interval (CI) to estimate the effectiveness of the strategies to improve retention. We assessed heterogeneity between trials. We applied GRADE to determine the certainty of the evidence within each comparison.

MAIN RESULTS

We identified 70 eligible papers that reported data from 81 retention trials. We included 69 studies with more than 100,000 participants in the final meta-analyses, of which 67 studies evaluated interventions aimed at trial participants and two evaluated interventions aimed at trial staff involved in retention. All studies were in health care and most aimed to improve postal questionnaire response. Interventions were categorised into broad comparison groups: Data collection; Participants; Sites and site staff; Central study management; and Study design. These intervention groups consisted of 52 comparisons, none of which were supported by high-certainty evidence as determined by GRADE assessment. There were four comparisons presenting moderate-certainty evidence, three supporting retention (self-sampling kits, monetary reward together with reminder or prenotification and giving a pen at recruitment) and one reducing retention (inclusion of a diary with usual follow-up compared to usual follow-up alone). Of the remaining studies, 20 presented GRADE low-certainty evidence and 28 presented very low-certainty evidence. Our findings do provide a priority list for future replication studies, especially with regard to comparisons that currently rely on a single study.

AUTHORS' CONCLUSIONS

Most of the interventions we identified aimed to improve retention in the form of postal questionnaire response. There were few evaluations of ways to improve participants returning to trial sites for trial follow-up. None of the comparisons are supported by high-certainty evidence. Comparisons in the review where the evidence certainty could be improved with the addition of well-done studies should be the focus for future evaluations.

Two-by-two factorial randomised study within a trial (SWAT) to evaluate strategies for follow-up in a randomised prevention trial

BACKGROUND

Failure to collect outcome data in randomised trials can result in bias and loss of statistical power. Further evaluations of strategies to increase retention are required. We assessed the effectiveness of two strategies for retention in a randomised prevention trial using a two-by-two factorial randomised study within a trial (SWAT).

METHODS

Parents of babies included in the host trial were randomised to (1) short message service (SMS) notification prior to sending questionnaires at 3, 6, 12 and 18 months versus no SMS notification and (2) a £10 voucher sent with the invitation letter for the primary follow-up visit at 24 months or given at the visit. The two co-primary outcomes were collection of host trial (1) questionnaire data at interim follow-up times and (2) primary outcome at 24 months during a home/clinic visit with a research nurse.

RESULTS

Between November 2014 and November 2016, 1394 participants were randomised: 350 to no SMS + voucher at visit, 345 to SMS + voucher at visit, 352 to no SMS + voucher before visit and 347 to SMS + voucher before visit. Overall questionnaire data was collected at interim follow-up times for 75% in both the group allocated to the prior SMS notification and the group allocated to no SMS notification (odds ratio (OR) SMS versus none 1.02, 95% CI 0.83 to 1.25). Host trial primary outcome data was collected at a visit for 557 (80%) allocated to the voucher before the visit in the invitation letter and for 566 (81%) whose parents were allocated to receive the voucher at the visit (OR before versus at visit 0.89, 95% CI 0.69 to 1.17).

CONCLUSION

There was no evidence of a difference in retention according to SMS notification or voucher timing. Future synthesis of SWAT results is required to be able to detect small but important incremental effects of retention strategies.

TRIAL REGISTRATION

ISRCTN registry, ID: ISRCTN21528841. Registered on 25 July 2014. SWAT Repository Store ID 25.