Systematic review of the cost-effectiveness of sample size maintenance programs in studies involving postal questionnaires reveals insufficient economic information

Most response-inducing strategies do not increase participation in observational studies: a systematic review and meta-analysis

OBJECTIVES

To evaluate response-inducing strategies for observational studies using health-related questionnaires or interviews.

STUDY DESIGN AND SETTING

We searched PubMed, EMBASE, CINAHL, PsycINFO, and Web of Science up to December 28, 2017. Studies evaluating the effect of a response-inducing strategy on participation rates of observational studies were included. For each strategy, we estimated pooled response ratios with 95% confidence intervals (CIs) in a Hartung-Knapp/Sidik-Jonkman random effects model with the final participation rate as outcome, stratified for type of participants and method of data collection.

RESULTS

The search yielded 168 eligible studies involving 367,616 potential participants and 33 strategies. Among patients, response-inducing strategies for paper-based questionnaires included unconditional monetary incentives (response ratio 1.15; 95% CI 1.09-1.21) and shorter questionnaires (1.04; 1.02-1.06). Among nonpatients, a personalized mode of delivery (1.47; 1.24-1.74), more expensive mailing type (1.25; 1.00-1.56), unconditional monetary incentives (1.24; 1.12-1.38), prenotification (1.12; 1.03-1.22), unconditional scratch lottery tickets (1.09; 1.01-1.18), and shorter questionnaires (1.06; 1.02-1.11) increased response rates to paper-based questionnaires. For Web-based questionnaires and interviews among nonpatients, response rates were increased by conditional lottery tickets (1.17; 1.02-1.34) and conditional monetary incentives (1.39; 1.01-1.91), respectively.

CONCLUSION

Although the majority of strategies evaluated were unsuccessful, some may increase response rates to observational studies, particularly among nonpatients.

Mode of delivery affected questionnaire response rates in a birth cohort study

Objectives

Cohort studies must collect data from their participants as economically as possible, while maintaining response rates. This randomized controlled trial investigated whether offering a choice of online or paper questionnaires resulted in improved response rates compared with offering online first.

Study Design and Setting

Eligible participants were young people in the Avon Longitudinal Study of Parents and Children (ALSPAC) study (born April 1, 1991, to December 31, 1992, in the Avon area). After exclusions, 8,795 participants were randomized. The “online first” group were invited to complete the questionnaire online. The “choice” group were also sent a paper questionnaire and offered a choice of completion method. The trial was embedded within routine data collection. The main outcome measure was the number of questionnaires returned. Data on costs were also collected.

Results

Those in the “online first” arm of the trial were less likely to return a questionnaire [adjusted odds ratio: 0.90; 95% confidence interval (CI): 0.82, 0.99]. The “choice” arm was more expensive (mean difference per participant £0.71; 95% CI: £0.65, £0.76). It cost an extra £47 to have one extra person to complete the questionnaire in the “choice” arm.

Conclusion

Offering a choice of completion methods (paper or online) for questionnaires in ALSPAC increased response rates but was more expensive than offering online first.

Lottery ticket was more effective than a prize draw in increasing questionnaire response among cancer survivors

OBJECTIVES

Compare the effect of financial incentives on response to a cancer survivors' postal questionnaire.

STUDY DESIGN AND SETTING

Prostate cancer survivors in Ireland, 1.5-18 years after diagnosis, were randomized to the (1) "lottery" arm [a € 1 lottery scratch card sent with the questionnaire (n = 2,413)] or (2) "prize" arm [entry into a draw on return of a completed questionnaire (n = 2,407)]. Impact of interventions on response overall and by survival period ("short term": < 5 years after diagnosis; "long term": ≥ 5 years after diagnosis) was compared as was cost-effectiveness.

RESULTS

Adjusted response rate was 54.4%. Response was higher among younger men (P < 0.001) and those with earlier stage disease (P = 0.002). A modest 2.6% higher response rate was observed in the lottery compared with the prize arm [multivariate relative risk (RR) = 1.06; 95% confidence interval (CI): 1.00, 1.11]. When stratified by survival period, higher response in the lottery arm was only observed among long-term survivors (multivariate RR = 1.10; 95% CI: 1.02, 1.19; short-term survivors: RR = 1.01; 95% CI: 0.94, 1.09). Costs per completed questionnaire were € 4.54 and € 3.57 for the lottery and prize arms, respectively. Compared with the prize arm, cost per additional questionnaire returned in the lottery arm was € 25.65.

CONCLUSION

Although more expensive, to optimize response to postal questionnaires among cancer survivors, researchers might consider inclusion of a lottery scratch card.

Meta-analysis of randomized controlled trials supports the use of incentives for inducing response to electronic health surveys

OBJECTIVES

The objective of this meta-analysis was to assess incentive effectiveness on response to electronic health surveys.

STUDY DESIGN AND SETTING

A systematic literature search of PubMed, Web of Science, CINAHL, EMBASE, and the Cochrane Library was performed from 1970 to March 2013. Two authors independently selected the trials, assessed methodological quality, and extracted data. Original authors were contacted for the missing information.

RESULTS

The search strategy yielded nine trials (including 29,463 participants in total) that met eligibility criteria. For each of the incentive strategies examined, a random-effects model was used because of significant heterogeneity, and results were summarized as pooled odds ratios (ORs). Compared with no incentive, the offer of an incentive was seen to have a beneficial effect on response (OR, 1.48; 95% CI: 1.29, 1.71). Specifically, the odds of response were more than doubled when a monetary incentive was used (OR, 2.43; 95% CI: 1.60, 3.69) and increased when nonmonetary incentives were used (OR, 1.33; 95% CI: 1.17, 1.51).

CONCLUSION

These findings indicate that health researchers using electronic surveys can improve the quality of their research by offering incentives to potential participants.