Lottery ticket was more effective than a prize draw in increasing questionnaire response among cancer survivors

Methods to increase response to postal and electronic questionnaires

BACKGROUND

Self-administered questionnaires are widely used to collect data in epidemiological research, but non-response reduces the effective sample size and can introduce bias. Finding ways to increase response to postal and electronic questionnaires would improve the quality of epidemiological research.

OBJECTIVES

To identify effective strategies to increase response to postal and electronic questionnaires.

SEARCH METHODS

We searched 14 electronic databases up to December 2021 and manually searched the reference lists of relevant trials and reviews. We contacted the authors of all trials or reviews to ask about unpublished trials; where necessary, we also contacted authors to confirm the methods of allocation used and to clarify results presented.

SELECTION CRITERIA

Randomised trials of methods to increase response to postal or electronic questionnaires. We assessed the eligibility of each trial using pre-defined criteria.

DATA COLLECTION AND ANALYSIS

We extracted data on the trial participants, the intervention, the number randomised to intervention and comparison groups and allocation concealment. For each strategy, we estimated pooled odds ratios (OR) and 95% confidence intervals (CI) in a random-effects model. We assessed evidence for selection bias using Egger's weighted regression method and Begg's rank correlation test and funnel plot. We assessed heterogeneity amongst trial odds ratios using a Chi2 test and quantified the degree of inconsistency between trial results using the I2 statistic.

MAIN RESULTS

Postal We found 670 eligible trials that evaluated over 100 different strategies of increasing response to postal questionnaires. We found substantial heterogeneity amongst trial results in half of the strategies. The odds of response almost doubled when: using monetary incentives (odds ratio (OR) 1.86; 95% confidence interval (CI) 1.73 to 1.99; heterogeneity I2 = 85%); using a telephone reminder (OR 1.96; 95% CI 1.03 to 3.74); and when clinical outcome questions were placed last (OR 2.05; 95% CI 1.00 to 4.24). The odds of response increased by about half when: using a shorter questionnaire (OR 1.58; 95% CI 1.40 to 1.78); contacting participants before sending questionnaires (OR 1.36; 95% CI 1.23 to 1.51; I2 = 87%); incentives were given with questionnaires (i.e. unconditional) rather than when given only after participants had returned their questionnaire (i.e. conditional on response) (OR 1.53; 95% CI 1.35 to 1.74); using personalised SMS reminders (OR 1.53; 95% CI 0.97 to 2.42); using a special (recorded) delivery service (OR 1.68; 95% CI 1.36 to 2.08; I2 = 87%); using electronic reminders (OR 1.60; 95% CI 1.10 to 2.33); using intensive follow-up (OR 1.69; 95% CI 0.93 to 3.06); using a more interesting/salient questionnaire (OR 1.73; 95% CI 1.12 to 2.66); and when mentioning an obligation to respond (OR 1.61; 95% CI 1.16 to 2.22). The odds of response also increased with: non-monetary incentives (OR 1.16; 95% CI 1.11 to 1.21; I2 = 80%); a larger monetary incentive (OR 1.24; 95% CI 1.15 to 1.33); a larger non-monetary incentive (OR 1.15; 95% CI 1.00 to 1.33); when a pen was included (OR 1.44; 95% CI 1.38 to 1.50); using personalised materials (OR 1.15; 95% CI 1.09 to 1.21; I2 = 57%); using a single-sided rather than a double-sided questionnaire (OR 1.13; 95% CI 1.02 to 1.25); using stamped return envelopes rather than franked return envelopes (OR 1.23; 95% CI 1.13 to 1.33; I2 = 69%), assuring confidentiality (OR 1.33; 95% CI 1.24 to 1.42); using first-class outward mailing (OR 1.11; 95% CI 1.02 to 1.21); and when questionnaires originated from a university (OR 1.32; 95% CI 1.13 to 1.54). The odds of response were reduced when the questionnaire included questions of a sensitive nature (OR 0.94; 95% CI 0.88 to 1.00). Electronic We found 88 eligible trials that evaluated over 30 different ways of increasing response to electronic questionnaires. We found substantial heterogeneity amongst trial results in half of the strategies. The odds of response tripled when: using a brief letter rather than a detailed letter (OR 3.26; 95% CI 1.79 to 5.94); and when a picture was included in an email (OR 3.05; 95% CI 1.84 to 5.06; I2 = 19%). The odds of response almost doubled when: using monetary incentives (OR 1.88; 95% CI 1.31 to 2.71; I2 = 79%); and using a more interesting topic (OR 1.85; 95% CI 1.52 to 2.26). The odds of response increased by half when: using non-monetary incentives (OR 1.60; 95% CI 1.25 to 2.05); using shorter e-questionnaires (OR 1.51; 95% CI 1.06 to 2.16; I2 = 94%); and using a more interesting e-questionnaire (OR 1.85; 95% CI 1.52 to 2.26). The odds of response increased by a third when: offering survey results as an incentive (OR 1.36; 95% CI 1.16 to 1.59); using a white background (OR 1.31; 95% CI 1.10 to 1.56); and when stressing the benefits to society of response (OR 1.38; 95% CI 1.07 to 1.78; I2 = 41%). The odds of response also increased with: personalised e-questionnaires (OR 1.24; 95% CI 1.17 to 1.32; I2 = 41%); using a simple header (OR 1.23; 95% CI 1.03 to 1.48); giving a deadline (OR 1.18; 95% CI 1.03 to 1.34); and by giving a longer time estimate for completion (OR 1.25; 95% CI 0.96 to 1.64). The odds of response were reduced when: "Survey" was mentioned in the e-mail subject (OR 0.81; 95% CI 0.67 to 0.97); when the email or the e-questionnaire was from a male investigator, or it included a male signature (OR 0.55; 95% CI 0.38 to 0.80); and by using university sponsorship (OR 0.84; 95%CI 0.69 to 1.01). The odds of response using a postal questionnaire were over twice those using an e-questionnaire (OR 2.33; 95% CI 2.25 to 2.42; I2 = 98%). Response also increased when: providing a choice of response mode (electronic or postal) rather than electronic only (OR 1.76 95% CI 1.67 to 1.85; I2 = 97%); and when administering the e-questionnaire by computer rather than by smartphone (OR 1.62 95% CI 1.36 to 1.94).

AUTHORS' CONCLUSIONS

Researchers using postal and electronic questionnaires can increase response using the strategies shown to be effective in this Cochrane review.

Behavioral economics implementation: Regret lottery improves mHealth patient study adherence

Background

Nonadherence to study protocols reduces the generalizability, validity, and statistical power of longitudinal studies.

Purpose

To determine whether an automated electronically-delivered regret lottery would improve adherence to an intensive mHealth self-monitoring protocol as part of a longitudinal observational study.

Methods

We enrolled 77 adults into a 52-week study requiring five daily ecologic momentary assessments (EMA) of stress and daily accelerometer use. We performed a pre/post single-arm study to evaluate the efficacy of a lottery intervention in improving adherence to this protocol. Midway through the study, participants were invited to enter a weekly regret lottery ($50 prize, expected value <$1) in which prize collection was contingent upon meeting adherence thresholds for the prior week. Study protocol adherence before and after lottery initiation were compared using mixed models repeated measures analysis of variance.

Results

62 participants consented to lottery participation. In the 12 weeks prior to lottery initiation, weekly adherence was declining (slope −1.4%/week). The weekly per-participant probability of adherence was higher after lottery initiation when comparing the 4-week (32% pre-lottery vs 50% post-lottery, p < 0.001), 8-week (37% vs 49%, p < 0.001), and 12-week periods (39% vs 45%, p = 0.001) before and after lottery initiation. However, the rate of decline in adherence over time was unchanged.

Conclusion

The implementation of an automated, electronically-delivered weekly regret lottery improved adherence with an intensive self-monitoring study protocol. Regret lotteries may represent a cost-effective tool to improve adherence and reduce bias caused by dropout or nonadherence.

Most response-inducing strategies do not increase participation in observational studies: a systematic review and meta-analysis

OBJECTIVES

To evaluate response-inducing strategies for observational studies using health-related questionnaires or interviews.

STUDY DESIGN AND SETTING

We searched PubMed, EMBASE, CINAHL, PsycINFO, and Web of Science up to December 28, 2017. Studies evaluating the effect of a response-inducing strategy on participation rates of observational studies were included. For each strategy, we estimated pooled response ratios with 95% confidence intervals (CIs) in a Hartung-Knapp/Sidik-Jonkman random effects model with the final participation rate as outcome, stratified for type of participants and method of data collection.

RESULTS

The search yielded 168 eligible studies involving 367,616 potential participants and 33 strategies. Among patients, response-inducing strategies for paper-based questionnaires included unconditional monetary incentives (response ratio 1.15; 95% CI 1.09-1.21) and shorter questionnaires (1.04; 1.02-1.06). Among nonpatients, a personalized mode of delivery (1.47; 1.24-1.74), more expensive mailing type (1.25; 1.00-1.56), unconditional monetary incentives (1.24; 1.12-1.38), prenotification (1.12; 1.03-1.22), unconditional scratch lottery tickets (1.09; 1.01-1.18), and shorter questionnaires (1.06; 1.02-1.11) increased response rates to paper-based questionnaires. For Web-based questionnaires and interviews among nonpatients, response rates were increased by conditional lottery tickets (1.17; 1.02-1.34) and conditional monetary incentives (1.39; 1.01-1.91), respectively.

CONCLUSION

Although the majority of strategies evaluated were unsuccessful, some may increase response rates to observational studies, particularly among nonpatients.

Establishing a population-based patient-reported outcomes study (PROMs) using national cancer registries across two jurisdictions: the Prostate Cancer Treatment, your experience (PiCTure) study

OBJECTIVE

To establish an international patient-reported outcomes (PROMs) study among prostate cancer survivors, up to 18 years postdiagnosis, in two countries with different healthcare systems and ethical frameworks.

DESIGN

A cross-sectional, postal survey of prostate cancer survivors sampled and recruited via two population-based cancer registries. Healthcare professionals (HCPs) evaluated patients for eligibility to participate. Questionnaires contained validated instruments to assess health-related quality of life and psychological well-being, including QLQ-C30, QLQ-PR25, EQ-5D-5L, 21-question Depression, Anxiety and Stress Scale (DASS-21) and the Decisional Regret Scale.

SETTING

Republic of Ireland (RoI) and Northern Ireland (NI).

PRIMARY OUTCOME MEASURES

Registration completeness, predictors of eligibility and response, data missingness, unweighted and weighted PROMs.

RESULTS

Prostate cancer registration was 80% (95% CI 75% to 84%) and 91% (95% CI 89% to 93%) complete 2 years postdiagnosis in NI and RoI, respectively. Of 12,322 survivors sampled from registries, 53% (n=6559) were classified as eligible following HCP screening. In the multivariate analysis, significant predictors of eligibility were: being ≤59 years of age at diagnosis (p<0.001), short-term survivor (<5 years postdiagnosis; p<0.001) and from RoI (p<0.001). 3348 completed the questionnaire, yielding a 54% adjusted response rate. 13% of men or their families called the study freephone with queries for assistance with questionnaire completion or to talk about their experience. Significant predictors of response in multivariate analysis were: being ≤59 years at diagnosis (p<0.001) and from RoI (p=0.016). Mean number of missing questions in validated instruments ranged from 0.12 (SD 0.71; EQ-5D-5L) to 3.72 (SD 6.30; QLQ-PR25). Weighted and unweighted mean EQ-5D-5L, QLQ-C30 and QLQ-PR25 scores were similar, as were the weighted and unweighted prevalences of depression, anxiety and distress.

CONCLUSIONS

It was feasible to perform PROMs studies across jurisdictions, using cancer registries as sampling frames; we amassed one of the largest, international, population-based data set of prostate cancer survivors. We highlight improvements which could inform future PROMs studies, including utilising general practitioners to assess eligibility and providing a freephone service.