Effects on subject response of information brochures and small cash incentives in a mail-based case-control study

Methods to increase response to postal and electronic questionnaires

BACKGROUND

Self-administered questionnaires are widely used to collect data in epidemiological research, but non-response reduces the effective sample size and can introduce bias. Finding ways to increase response to postal and electronic questionnaires would improve the quality of epidemiological research.

OBJECTIVES

To identify effective strategies to increase response to postal and electronic questionnaires.

SEARCH METHODS

We searched 14 electronic databases up to December 2021 and manually searched the reference lists of relevant trials and reviews. We contacted the authors of all trials or reviews to ask about unpublished trials; where necessary, we also contacted authors to confirm the methods of allocation used and to clarify results presented.

SELECTION CRITERIA

Randomised trials of methods to increase response to postal or electronic questionnaires. We assessed the eligibility of each trial using pre-defined criteria.

DATA COLLECTION AND ANALYSIS

We extracted data on the trial participants, the intervention, the number randomised to intervention and comparison groups and allocation concealment. For each strategy, we estimated pooled odds ratios (OR) and 95% confidence intervals (CI) in a random-effects model. We assessed evidence for selection bias using Egger's weighted regression method and Begg's rank correlation test and funnel plot. We assessed heterogeneity amongst trial odds ratios using a Chi2 test and quantified the degree of inconsistency between trial results using the I2 statistic.

MAIN RESULTS

Postal We found 670 eligible trials that evaluated over 100 different strategies of increasing response to postal questionnaires. We found substantial heterogeneity amongst trial results in half of the strategies. The odds of response almost doubled when: using monetary incentives (odds ratio (OR) 1.86; 95% confidence interval (CI) 1.73 to 1.99; heterogeneity I2 = 85%); using a telephone reminder (OR 1.96; 95% CI 1.03 to 3.74); and when clinical outcome questions were placed last (OR 2.05; 95% CI 1.00 to 4.24). The odds of response increased by about half when: using a shorter questionnaire (OR 1.58; 95% CI 1.40 to 1.78); contacting participants before sending questionnaires (OR 1.36; 95% CI 1.23 to 1.51; I2 = 87%); incentives were given with questionnaires (i.e. unconditional) rather than when given only after participants had returned their questionnaire (i.e. conditional on response) (OR 1.53; 95% CI 1.35 to 1.74); using personalised SMS reminders (OR 1.53; 95% CI 0.97 to 2.42); using a special (recorded) delivery service (OR 1.68; 95% CI 1.36 to 2.08; I2 = 87%); using electronic reminders (OR 1.60; 95% CI 1.10 to 2.33); using intensive follow-up (OR 1.69; 95% CI 0.93 to 3.06); using a more interesting/salient questionnaire (OR 1.73; 95% CI 1.12 to 2.66); and when mentioning an obligation to respond (OR 1.61; 95% CI 1.16 to 2.22). The odds of response also increased with: non-monetary incentives (OR 1.16; 95% CI 1.11 to 1.21; I2 = 80%); a larger monetary incentive (OR 1.24; 95% CI 1.15 to 1.33); a larger non-monetary incentive (OR 1.15; 95% CI 1.00 to 1.33); when a pen was included (OR 1.44; 95% CI 1.38 to 1.50); using personalised materials (OR 1.15; 95% CI 1.09 to 1.21; I2 = 57%); using a single-sided rather than a double-sided questionnaire (OR 1.13; 95% CI 1.02 to 1.25); using stamped return envelopes rather than franked return envelopes (OR 1.23; 95% CI 1.13 to 1.33; I2 = 69%), assuring confidentiality (OR 1.33; 95% CI 1.24 to 1.42); using first-class outward mailing (OR 1.11; 95% CI 1.02 to 1.21); and when questionnaires originated from a university (OR 1.32; 95% CI 1.13 to 1.54). The odds of response were reduced when the questionnaire included questions of a sensitive nature (OR 0.94; 95% CI 0.88 to 1.00). Electronic We found 88 eligible trials that evaluated over 30 different ways of increasing response to electronic questionnaires. We found substantial heterogeneity amongst trial results in half of the strategies. The odds of response tripled when: using a brief letter rather than a detailed letter (OR 3.26; 95% CI 1.79 to 5.94); and when a picture was included in an email (OR 3.05; 95% CI 1.84 to 5.06; I2 = 19%). The odds of response almost doubled when: using monetary incentives (OR 1.88; 95% CI 1.31 to 2.71; I2 = 79%); and using a more interesting topic (OR 1.85; 95% CI 1.52 to 2.26). The odds of response increased by half when: using non-monetary incentives (OR 1.60; 95% CI 1.25 to 2.05); using shorter e-questionnaires (OR 1.51; 95% CI 1.06 to 2.16; I2 = 94%); and using a more interesting e-questionnaire (OR 1.85; 95% CI 1.52 to 2.26). The odds of response increased by a third when: offering survey results as an incentive (OR 1.36; 95% CI 1.16 to 1.59); using a white background (OR 1.31; 95% CI 1.10 to 1.56); and when stressing the benefits to society of response (OR 1.38; 95% CI 1.07 to 1.78; I2 = 41%). The odds of response also increased with: personalised e-questionnaires (OR 1.24; 95% CI 1.17 to 1.32; I2 = 41%); using a simple header (OR 1.23; 95% CI 1.03 to 1.48); giving a deadline (OR 1.18; 95% CI 1.03 to 1.34); and by giving a longer time estimate for completion (OR 1.25; 95% CI 0.96 to 1.64). The odds of response were reduced when: "Survey" was mentioned in the e-mail subject (OR 0.81; 95% CI 0.67 to 0.97); when the email or the e-questionnaire was from a male investigator, or it included a male signature (OR 0.55; 95% CI 0.38 to 0.80); and by using university sponsorship (OR 0.84; 95%CI 0.69 to 1.01). The odds of response using a postal questionnaire were over twice those using an e-questionnaire (OR 2.33; 95% CI 2.25 to 2.42; I2 = 98%). Response also increased when: providing a choice of response mode (electronic or postal) rather than electronic only (OR 1.76 95% CI 1.67 to 1.85; I2 = 97%); and when administering the e-questionnaire by computer rather than by smartphone (OR 1.62 95% CI 1.36 to 1.94).

AUTHORS' CONCLUSIONS

Researchers using postal and electronic questionnaires can increase response using the strategies shown to be effective in this Cochrane review.

Incentivizing optimal risk map use for Triatoma infestans surveillance in urban environments

In Arequipa, Peru, a large-scale vector control campaign has successfully reduced urban infestations of the Chagas disease vector, Triatoma infestans. In addition to preventing new infections with Trypanosoma cruzi (etiological agent of Chagas disease), the campaign produced a wealth of information about the distribution and density of vector infestations. We used these data to create vector infestation risk maps for the city in order to target the last few remaining infestations, which are unevenly distributed and difficult to pinpoint. Our maps, which are provided on a mobile app, display color-coded, individual house-level estimates of T. infestans infestation risk. Entomologic surveillance personnel can use the maps to select homes to inspect based on estimated risk of infestation, as well as keep track of which parts of a given neighborhood they have inspected to ensure even surveillance throughout the zone. However, the question then becomes, how do we encourage surveillance personnel to actually use these two functionalities of the risk map? As such, we carried out a series of rolling trials to test different incentive schemes designed to encourage the following two behaviors by entomologic surveillance personnel in Arequipa: (i) preferential inspections of homes shown as high risk on the maps, and (ii) even surveillance across the geographical distribution of a given area, which we term, 'spatial coverage.' These two behaviors together constituted what we termed, 'optimal map use.' We found that several incentives resulted in one of the two target behaviors, but just one incentive scheme based on the game of poker resulted in optimal map use. This poker-based incentive structure may be well-suited to improve entomological surveillance activities and other complex multi-objective tasks.

Randomized control trial of unconditional versus conditional incentives to increase study enrollment rates in participants at increased risk of lung cancer

INTRODUCTION

Understanding how incentives and their timing influence study enrollment rates is important to efficient study design and increasing the generalizability of findings. This 2-arm, parallel randomized trial evaluated how conditional vs. unconditional mailed incentives of a $20 gift card affected study enrollment in a sample of participants screened for lung cancer screening.

METHODS

Eligible participants included Black and White adults who underwent lung cancer screening with low-dose CT and had negative screening results at two North Carolina imaging facilities in 2018. We used a stratified randomization scheme, by sex and race, to assign incentive type (conditional vs. unconditional). We used the Tailored Design Method with six points of mailed contact to engage participants. We compared study enrollment rates using chi-square tests and logistic regression analyses.

RESULTS

After adjusting for sex, race, age, smoking status, participant residence, and screening site, participants who received unconditional incentives were 74% more likely to enroll than those who received conditional incentives (adjusted OR = 1.74 (95% CI: 1.01, 3.00).

CONCLUSIONS

Type of incentive can play a role in increasing study enrollment, especially mailed surveys that target individuals who currently or previously smoked. Unconditional incentives may be worth the initial cost to engage study participants.

Association between response rates and monetary incentives in sample study: a systematic review and meta-analysis

OBJECTIVE

To investigate the effect of monetary incentive and the dose-response relationship of participants' response rates in surveys.

METHODS

Three databases were searched for randomised controlled trials (RCTs) that investigated the effect of monetary incentives on participants' first and final response rates. First response is defined as the responses after the participant was initially contacted and final response is defined as the responses after several reminders were sent. The potential dose-response relationship of the amount of monetary incentive on the relative response rate (RRR) was established by fitting a restricted cubic spline function based on the robust-error meta-regression model.

RESULTS

105 RCTs were identified. The first RRR increased by 49% (RRR=1.49; 95% CI 1.29 to 1.72) when monetary incentives were provided. Dose-response analysis revealed that an amount between US$6.25 and US$8 had the maximum effect on increasing the first response rate. On average, the final RRR increased almost by 20% (RRR=1.18; 95% CI 1.11 to 1.25) with monetary incentive compared to no-monetary incentive. An amount between US$10 and US$15 had the maximum effect on the final response rate, with an increase in the final RRR of 34% (RRR=1.34; 95% CI 1.19 to 1.51). There was a significant increase in the response rate when two or more reminders were sent.

CONCLUSION

Monetary incentives and reminders improve the response rates. Future studies need to consider providing monetary incentives and sending at least two reminders to increase the response rate and reduce the chances of non-response bias.

The feasibility of web surveys for obtaining patient-reported outcomes from cancer survivors: a randomized experiment comparing survey modes and brochure enclosures

Background

Central cancer registries are often used to survey population-based samples of cancer survivors. These surveys are typically administered via paper or telephone. In most populations, web surveys obtain much lower response rates than paper surveys. This study assessed the feasibility of web surveys for collecting patient-reported outcomes via a central cancer registry.

Methods

Potential participants were sampled from Utah Cancer Registry records. Sample members were randomly assigned to receive a web or paper survey, and then randomized to either receive or not receive an informative brochure describing the cancer registry. We calculated adjusted risk ratios with 95% confidence intervals to compare response likelihood and the demographic profile of respondents across study arms.

Results

The web survey response rate (43.2%) was lower than the paper survey (50.4%), but this difference was not statistically significant (adjusted risk ratio = 0.88, 95% confidence interval = 0.72, 1.07). The brochure also did not significantly influence the proportion responding (adjusted risk ratio = 1.03, 95% confidence interval = 0.85, 1.25). There were few differences in the demographic profiles of respondents across the survey modes. Older age increased likelihood of response to a paper questionnaire but not a web questionnaire.

Conclusions

Web surveys of cancer survivors are feasible without significantly influencing response rates, but providing a paper response option may be advisable particularly when surveying older individuals. Further examination of the varying effects of brochure enclosures across different survey modes is warranted.

The feasibility of web surveys for obtaining patient-reported outcomes from cancer survivors: a randomized experiment comparing survey modes and brochure enclosures

BACKGROUND

Central cancer registries are often used to survey population-based samples of cancer survivors. These surveys are typically administered via paper or telephone. In most populations, web surveys obtain much lower response rates than paper surveys. This study assessed the feasibility of web surveys for collecting patient-reported outcomes via a central cancer registry.

METHODS

Potential participants were sampled from Utah Cancer Registry records. Sample members were randomly assigned to receive a web or paper survey, and then randomized to either receive or not receive an informative brochure describing the cancer registry. We calculated adjusted risk ratios with 95% confidence intervals to compare response likelihood and the demographic profile of respondents across study arms.

RESULTS

The web survey response rate (43.2%) was lower than the paper survey (50.4%), but this difference was not statistically significant (adjusted risk ratio = 0.88, 95% confidence interval = 0.72, 1.07). The brochure also did not significantly influence the proportion responding (adjusted risk ratio = 1.03, 95% confidence interval = 0.85, 1.25). There were few differences in the demographic profiles of respondents across the survey modes. Older age increased likelihood of response to a paper questionnaire but not a web questionnaire.

CONCLUSIONS

Web surveys of cancer survivors are feasible without significantly influencing response rates, but providing a paper response option may be advisable particularly when surveying older individuals. Further examination of the varying effects of brochure enclosures across different survey modes is warranted.

Predictors of Response Outcomes for Research Recruitment Through a Central Cancer Registry: Evidence From 17 Recruitment Efforts for Population-Based Studies

When recruiting research participants through central cancer registries, high response fractions help ensure population-based representation. We conducted multivariable mixed-effects logistic regression to identify case and study characteristics associated with making contact with and obtaining cooperation of Utah cancer cases using data from 17 unique recruitment efforts undertaken by the Utah Cancer Registry (2007-2016) on behalf of the following studies: A Population-Based Childhood Cancer Survivors Cohort Study in Utah, Comparative Effectiveness Analysis of Surgery and Radiation for Prostate Cancer (CEASAR Study), Costs and Benefits of Follow-up Care for Adolescent and Young Adult Cancers, Study of Exome Sequencing for Head and Neck Cancer Susceptibility Genes, Genetic Epidemiology of Chronic Lymphocytic Leukemia, Impact of Remote Familial Colorectal Cancer Risk Assessment and Counseling (Family CARE Project), Massively Parallel Sequencing for Familial Colon Cancer Genes, Medullary Thyroid Carcinoma (MTC) Surveillance Study, Osteosarcoma Surveillance Study, Prostate Cancer Outcomes Study, Risk Education and Assessment for Cancer Heredity Project (REACH Project), Study of Shared Genomic Segment Analysis and Tumor Subtyping in High-Risk Breast-Cancer Gene Pedigrees, Study of Shared Genomic Segment Analysis for Localizing Multiple Myeloma Genes. Characteristics associated with lower odds of contact included Hispanic ethnicity (odds ratio (OR) = 0.34, 95% confidence interval (CI): 0.27, 0.41), nonwhite race (OR = 0.46, 95% CI: 0.35, 0.60), and younger age at contact. Years since diagnosis was inversely associated with making contact. Nonwhite race and age ≥60 years had lower odds of cooperation. Study features with lower odds of cooperation included longitudinal design (OR = 0.50, 95% CI: 0.41, 0.61) and study brochures (OR = 0.70, 95% CI: 0.54, 0.90). Increased odds of cooperation were associated with including a questionnaire (OR = 3.19, 95% CI: 1.54, 6.59), postage stamps (OR = 1.60, 95% CI: 1.21, 2.12), and incentives (OR = 1.62, 95% CI: 1.02, 2.57). Among cases not responding after the first contact, odds of eventual response were lower when >10 days elapsed before subsequent contact (OR = 0.71, 95% CI: 0.59, 0.85). Obtaining high response is challenging, but study features identified in this analysis support better results when recruiting through central cancer registries.

Most response-inducing strategies do not increase participation in observational studies: a systematic review and meta-analysis

OBJECTIVES

To evaluate response-inducing strategies for observational studies using health-related questionnaires or interviews.

STUDY DESIGN AND SETTING

We searched PubMed, EMBASE, CINAHL, PsycINFO, and Web of Science up to December 28, 2017. Studies evaluating the effect of a response-inducing strategy on participation rates of observational studies were included. For each strategy, we estimated pooled response ratios with 95% confidence intervals (CIs) in a Hartung-Knapp/Sidik-Jonkman random effects model with the final participation rate as outcome, stratified for type of participants and method of data collection.

RESULTS

The search yielded 168 eligible studies involving 367,616 potential participants and 33 strategies. Among patients, response-inducing strategies for paper-based questionnaires included unconditional monetary incentives (response ratio 1.15; 95% CI 1.09-1.21) and shorter questionnaires (1.04; 1.02-1.06). Among nonpatients, a personalized mode of delivery (1.47; 1.24-1.74), more expensive mailing type (1.25; 1.00-1.56), unconditional monetary incentives (1.24; 1.12-1.38), prenotification (1.12; 1.03-1.22), unconditional scratch lottery tickets (1.09; 1.01-1.18), and shorter questionnaires (1.06; 1.02-1.11) increased response rates to paper-based questionnaires. For Web-based questionnaires and interviews among nonpatients, response rates were increased by conditional lottery tickets (1.17; 1.02-1.34) and conditional monetary incentives (1.39; 1.01-1.91), respectively.

CONCLUSION

Although the majority of strategies evaluated were unsuccessful, some may increase response rates to observational studies, particularly among nonpatients.

Using incentives in surveys of cancer patients: do "best practices" apply?

PURPOSE

Surveys of cancer patients are an important means of collecting data necessary to improve cancer prevention and control. However, health surveys generally are characterized by declining response rates, with incentives often employed to encourage participation. While successful, magnitude of effect is partially dependent upon situational characteristics of respondents, including health status. Given the health challenges experienced by cancer patients, it is unclear to what extent incentives can improve survey participation. In this study, we examine the effectiveness of monetary and non-monetary incentives in improving response to cancer patient surveys.

METHODS

We reviewed the available experimental literature regarding efforts to improve response rates among cancer patients/survivors via incentives. Relevant studies were identified through searches of the MEDLINE, PubMed, and PsychINFO databases from 1975 to 2012. Seed sources (e.g., Cancer Causes & Control, Cancer Epidemiology, Biomarkers & Prevention, and BMC Medical Research Methodology) were also referenced extensively in order to establish a comprehensive set of studies.

RESULTS

Although limited, evidence does suggest that token incentives may be less effective for improving survey participation among cancer patients, relative to other population groups. These results are contrary to well-established evidence regarding the efficacy of incentives in improving survey participation generally. Potential reasons why incentives may be less effective in this population are explored.

CONCLUSIONS

While more research is necessary, results suggest that survey research strategies targeting cancer patients be purposively designed in a manner that gives consideration to the distress associated with the condition, including selection of alternative strategies to improve response.

A Review of Paying Research Participants: It's Time to Move Beyond the ethical Debate

CURRENT REGULATORY GUIDELINES REQUIRE the ethical review committee to consider one question when evaluating payment: Is the payment to the participant undue or coercive? Although this is a seemingly simple question, determining appropriate payment involves a series of complex issues. There is limited empirical knowledge to assist with this determination and little consensus on which elements of a study should be considered in making these decisions. For example, should the culture of the study population or the potential risks and benefits of the research be considered in the selection of appropriate payment? Following a review of national and international guidelines, the concerns and benefits of paying research participants are presented, and prior ethical debate is outlined. The current research literature on the practice of paying participants and the impact of payment on participants and study integrity are reviewed. Finally, given continued debate with limited data to help determine best practices, a research agenda is proposed to assist in the development of an empirical basis to aid investigators and ethical review committees in making appropriate decisions about payment to research participants.

A review of how to conduct a surgical survey using a questionnaire

Health surveys using questionnaires facilitate the acquisition of information on the knowledge, behaviour, attitudes, perceptions and clinical history of a selected population. Their internal and external validities are threatened by poor design and low response rates. Numerous studies have investigated survey design and administration but care should be taken when generalising findings in different clinical and cultural settings. The current evidence-base suggests that no single mode of survey administration, such as postal, electronic or telephone, is superior to another. Whilst there is no evidence of an ideal response rate relationship to survey validity, response rates can be enhanced by including monetary incentives, providing a time cue, and repeat contact with non-responders. Unlike other modes of experimental data collection, few guidelines currently exist for survey and questionnaire design and response rate should not be considered a direct measure of a survey's quality.

A comparison of small monetary incentives to convert survey non-respondents: a randomized control trial

BACKGROUND

Maximizing response rates is critically important in order to provide the most generalizable and unbiased research results. High response rates reduce the chance of respondents being systematically different from non-respondents, and thus, reduce the risk of results not truly reflecting the study population. Monetary incentives are often used to improve response rates, but little is known about whether larger incentives improve response rates in those who previously have been unenthusiastic about participating in research. In this study we compared the response rates and cost-effectiveness of a $5 versus $2 monetary incentive accompanying a short survey mailed to patients who did not respond or refused to participate in research study with a face-to-face survey.

METHODS

1,328 non-responders were randomly assigned to receive $5 or $2 and a short, 10-question survey by mail. Reminder postcards were sent to everyone; those not returning the survey were sent a second survey without incentive. Overall response rates, response rates by incentive condition, and odds of responding to the larger incentive were calculated. Total costs (materials, postage, and labor) and incremental cost-effectiveness ratios were also calculated and compared by incentive condition.

RESULTS

After the first mailing, the response rate within the $5 group was significantly higher (57.8% vs. 47.7%, p<.001); after the second mailing, the difference narrowed by 80%, resulting in a non-significant difference in cumulative rates between the $5 and $2 groups (67.3% vs. 65.4%, respectively, p=.47). Regardless of incentive or number of contacts, respondents were significantly more likely to be male, white, married, and 50-75 years old. Total costs were higher with the larger versus smaller incentive ($13.77 versus $9.95 per completed survey).

CONCLUSIONS

A $5 incentive provides a significantly higher response rate than a $2 incentive if only one survey mailing is used but not if two survey mailings are used.

Why do we pay? A national survey of investigators and IRB chairpersons

The principle that payment to participants should not be undue or coercive is the consensus of international and national guidelines and ethical debates; however, what this means in practice is unclear. This study determined the attitudes and practices of IRB chairpersons and investigators regarding participant payment. One thousand six hundred investigators and 1900 IRB chairpersons received an invitation to participate in a web-based survey. Four hundred and fifty-five investigators (28.3%) and 395 IRB chairpersons (18.6%) responded. The survey was designed to gather considerations that govern payment determination and practical application of these considerations in hypothetical case studies. The survey asked best answer, multiple choice, and open text questions. Short hypothetical case scenarios where presented, and participants were asked to rate factors in the study that might impact payment and then determine their recommended payment. A predictive model was developed for each case to determine factors which affected payment. Although compensation was the primary reason given to justify payment by both investigators and IRB chairpersons, the cases suggested that, in practice, payment is often guided by incentive, as shown by the impact of anticipated difficulty recruiting, inconvenience, and risk in determining payment. Payment models varied by type of study. Ranges for recommended payments by both groups for different types of procedures and studies are presented.

Blood spots as an alternative to whole blood collection and the effect of a small monetary incentive to increase participation in genetic association studies

Background

Collection of buccal cells from saliva for DNA extraction offers a less invasive and convenient alternative to venipuncture blood collection that may increase participation in genetic epidemiologic studies. However, dried blood spot collection, which is also a convenient method, offers a means of collecting peripheral blood samples from which analytes in addition to DNA can be obtained.

Methods

To determine if offering blood spot collection would increase participation in genetic epidemiologic studies, we conducted a study of collecting dried blood spot cards by mail from a sample of female cancer cases (n = 134) and controls (n = 256) who were previously selected for a breast cancer genetics study and declined to provide a venipuncture blood sample. Participants were also randomized to receive either a $2.00 bill or no incentive with the blood spot collection kits.

Results

The average time between the venipuncture sample refusal and recruitment for the blood spot collection was 4.4 years. Thirty-seven percent of cases and 28% of controls provided a dried blood spot card. While the incentive was not associated with participation among controls (29% for $2.00 incentive vs. 26% for no incentive, p = 0.6), it was significantly associated with participation among the breast cancer cases (48% vs. 27%, respectively, p = 0.01). There did not appear to be any bias in response since no differences between cases and controls and incentive groups were observed when examining several demographic, work history and radiation exposure variables.

Conclusion

This study demonstrates that collection of dried blood spot cards in addition to venipuncture blood samples may be a feasible method to increase participation in genetic case-control studies.

Methods to increase response to postal and electronic questionnaires

BACKGROUND

Postal and electronic questionnaires are widely used for data collection in epidemiological studies but non-response reduces the effective sample size and can introduce bias. Finding ways to increase response to postal and electronic questionnaires would improve the quality of health research.

OBJECTIVES

To identify effective strategies to increase response to postal and electronic questionnaires.

SEARCH STRATEGY

We searched 14 electronic databases to February 2008 and manually searched the reference lists of relevant trials and reviews, and all issues of two journals. We contacted the authors of all trials or reviews to ask about unpublished trials. Where necessary, we also contacted authors to confirm methods of allocation used and to clarify results presented. We assessed the eligibility of each trial using pre-defined criteria.

SELECTION CRITERIA

Randomised controlled trials of methods to increase response to postal or electronic questionnaires.

DATA COLLECTION AND ANALYSIS

We extracted data on the trial participants, the intervention, the number randomised to intervention and comparison groups and allocation concealment. For each strategy, we estimated pooled odds ratios (OR) and 95% confidence intervals (CI) in a random-effects model. We assessed evidence for selection bias using Egger's weighted regression method and Begg's rank correlation test and funnel plot. We assessed heterogeneity among trial odds ratios using a Chi(2) test and the degree of inconsistency between trial results was quantified using the I(2) statistic.

MAIN RESULTS

PostalWe found 481 eligible trials. The trials evaluated 110 different ways of increasing response to postal questionnaires. We found substantial heterogeneity among trial results in half of the strategies. The odds of response were at least doubled using monetary incentives (odds ratio 1.87; 95% CI 1.73 to 2.04; heterogeneity P < 0.00001, I(2) = 84%), recorded delivery (1.76; 95% CI 1.43 to 2.18; P = 0.0001, I(2) = 71%), a teaser on the envelope - e.g. a comment suggesting to participants that they may benefit if they open it (3.08; 95% CI 1.27 to 7.44) and a more interesting questionnaire topic (2.00; 95% CI 1.32 to 3.04; P = 0.06, I(2) = 80%). The odds of response were substantially higher with pre-notification (1.45; 95% CI 1.29 to 1.63; P < 0.00001, I(2) = 89%), follow-up contact (1.35; 95% CI 1.18 to 1.55; P < 0.00001, I(2) = 76%), unconditional incentives (1.61; 1.36 to 1.89; P < 0.00001, I(2) = 88%), shorter questionnaires (1.64; 95% CI 1.43 to 1.87; P < 0.00001, I(2) = 91%), providing a second copy of the questionnaire at follow up (1.46; 95% CI 1.13 to 1.90; P < 0.00001, I(2) = 82%), mentioning an obligation to respond (1.61; 95% CI 1.16 to 2.22; P = 0.98, I(2) = 0%) and university sponsorship (1.32; 95% CI 1.13 to 1.54; P < 0.00001, I(2) = 83%). The odds of response were also increased with non-monetary incentives (1.15; 95% CI 1.08 to 1.22; P < 0.00001, I(2) = 79%), personalised questionnaires (1.14; 95% CI 1.07 to 1.22; P < 0.00001, I(2) = 63%), use of hand-written addresses (1.25; 95% CI 1.08 to 1.45; P = 0.32, I(2) = 14%), use of stamped return envelopes as opposed to franked return envelopes (1.24; 95% CI 1.14 to 1.35; P < 0.00001, I(2) = 69%), an assurance of confidentiality (1.33; 95% CI 1.24 to 1.42) and first class outward mailing (1.11; 95% CI 1.02 to 1.21; P = 0.78, I(2) = 0%). The odds of response were reduced when the questionnaire included questions of a sensitive nature (0.94; 95% CI 0.88 to 1.00; P = 0.51, I(2) = 0%).ElectronicWe found 32 eligible trials. The trials evaluated 27 different ways of increasing response to electronic questionnaires. We found substantial heterogeneity among trial results in half of the strategies. The odds of response were increased by more than a half using non-monetary incentives (1.72; 95% CI 1.09 to 2.72; heterogeneity P < 0.00001, I(2) = 95%), shorter e-questionnaires (1.73; 1.40 to 2.13; P = 0.08, I(2) = 68%), including a statement that others had responded (1.52; 95% CI 1.36 to 1.70), and a more interesting topic (1.85; 95% CI 1.52 to 2.26). The odds of response increased by a third using a lottery with immediate notification of results (1.37; 95% CI 1.13 to 1.65), an offer of survey results (1.36; 95% CI 1.15 to 1.61), and using a white background (1.31; 95% CI 1.10 to 1.56). The odds of response were also increased with personalised e-questionnaires (1.24; 95% CI 1.17 to 1.32; P = 0.07, I(2) = 41%), using a simple header (1.23; 95% CI 1.03 to 1.48), using textual representation of response categories (1.19; 95% CI 1.05 to 1.36), and giving a deadline (1.18; 95% CI 1.03 to 1.34). The odds of response tripled when a picture was included in an e-mail (3.05; 95% CI 1.84 to 5.06; P = 0.27, I(2) = 19%). The odds of response were reduced when "Survey" was mentioned in the e-mail subject line (0.81; 95% CI 0.67 to 0.97; P = 0.33, I(2) = 0%), and when the e-mail included a male signature (0.55; 95% CI 0.38 to 0.80; P = 0.96, I(2) = 0%).

AUTHORS' CONCLUSIONS

Health researchers using postal and electronic questionnaires can increase response using the strategies shown to be effective in this systematic review.

Methods to increase response rates to postal questionnaires

BACKGROUND

Postal questionnaires are widely used for data collection in epidemiological studies but non-response reduces the effective sample size and can introduce bias. Finding ways to increase response rates to postal questionnaires would improve the quality of health research.

OBJECTIVES

To identify effective strategies to increase response rates to postal questionnaires.

SEARCH STRATEGY

We aimed to find all randomised controlled trials of strategies to increase response rates to postal questionnaires. We searched 14 electronic databases to February 2003 and manually searched the reference lists of relevant trials and reviews, and all issues of two journals. We contacted the authors of all trials or reviews to ask about unpublished trials. Where necessary, authors were also contacted to confirm methods of allocation used and to clarify results presented. We assessed the eligibility of each trial using pre-defined criteria.

SELECTION CRITERIA

Randomised controlled trials of methods to increase response rates to postal questionnaires.

DATA COLLECTION AND ANALYSIS

We extracted data on the trial participants, the intervention, the number randomised to intervention and comparison groups and allocation concealment. For each strategy, we estimated pooled odds ratios and 95% confidence intervals in a random effects model. Evidence for selection bias was assessed using Egger's weighted regression method and Begg's rank correlation test and funnel plot. Heterogeneity among trial odds ratios was assessed using a chi-square test at a 5% significance level and the degree of inconsistency between trial results was quantified using I(2).

MAIN RESULTS

We found 372 eligible trials. The trials evaluated 98 different ways of increasing response rates to postal questionnaires and for 62 of these the combined trials included over 1,000 participants. We found substantial heterogeneity among trial results in half of the strategies. The odds of response were at least doubled using monetary incentives (odds ratio 1.99, 95% CI 1.81 to 2.18; heterogeneity p<0.00001, I(2)=78%), recorded delivery (2.04, 1.60 to 2.61; p=0.0004, I(2)=69%), a teaser on the envelope - e.g. a comment suggesting to participants that they may benefit if they open it (3.08, 1.27 to 7.44) and a more interesting questionnaire topic (2.44, 1.99 to 3.01; p=0.74, I(2)=0%). The odds of response were substantially higher with pre-notification (1.50, 1.29 to 1.74; p<0.00001, I(2)=90%), follow-up contact (1.44, 1.25 to 1.65; p<0.0001, I(2)=68%), unconditional incentives (1.61, 1.27 to 2.04; p<0.00001, I(2)=91%), shorter questionnaires (1.73, 1.47 to 2.03; p<0.00001, I(2)=93%), providing a second copy of the questionnaire at follow-up (1.51, 1.13 to 2.00; p<0.00001, I(2)=83%), mentioning an obligation to respond (1.61, 1.16 to 2.22; p=0.98, I(2)=0%) and university sponsorship (1.32, 1.13 to 1.54; p<0.00001, I(2)=83%). The odds of response were also increased with non-monetary incentives (1.13, 1.07 to 1.21; p<0.00001, I(2)=71%), personalised questionnaires (1.16, 1.07 to 1.26; p<0.00001, I(2)=67%), use of coloured as opposed to blue or black ink (1.39, 1.16 to 1.67), use of stamped return envelopes as opposed to franked return envelopes (1.29, 1.18 to 1.42; p<0.00001, I(2)=72%), an assurance of confidentiality (1.33, 1.24 to 1.42) and first class outward mailing (1.12, 1.02 to 1.23). The odds of response were reduced when the questionnaire included questions of a sensitive nature (0.94, 0.88 to 1.00; p=0.51, I(2)=0%), when questionnaires began with the most general questions (0.80, 0.67 to 0.96), or when participants were offered the opportunity to opt out of the study (0.76, 0.65 to 0.89; p=0.46, I(2)=0%).

AUTHORS' CONCLUSIONS

Health researchers using postal questionnaires can increase response rates using the strategies shown to be effective in this systematic review.

Maximising response to postal questionnaires--a systematic review of randomised trials in health research

Background

Postal self-completion questionnaires offer one of the least expensive modes of collecting patient based outcomes in health care research. The purpose of this review is to assess the efficacy of methods of increasing response to postal questionnaires in health care studies on patient populations.

Methods

The following databases were searched: Medline, Embase, CENTRAL, CDSR, PsycINFO, NRR and ZETOC. Reference lists of relevant reviews and relevant journals were hand searched. Inclusion criteria were randomised trials of strategies to improve questionnaire response in health care research on patient populations. Response rate was defined as the percentage of questionnaires returned after all follow-up efforts. Study quality was assessed by two independent reviewers. The Mantel-Haenszel method was used to calculate the pooled odds ratios.

Results

Thirteen studies reporting fifteen trials were included. Implementation of reminder letters and telephone contact had the most significant effect on response rates (odds ratio 3.7, 95% confidence interval 2.30 to 5.97 p = <0.00001). Shorter questionnaires also improved response rates to a lesser degree (odds ratio 1.4, 95% confidence interval 1.19 to 1.54). No evidence was found that incentives, re-ordering of questions or including an information brochure with the questionnaire confer any additional advantage.

Conclusion

Implementing repeat mailing strategies and/or telephone reminders may improve response to postal questionnaires in health care research. Making the questionnaire shorter may also improve response rates. There is a lack of evidence to suggest that incentives are useful. In the context of health care research all strategies to improve response to postal questionnaires require further evaluation.

A case?control study of occupational risk factors for bladder cancer in Canada

OBJECTIVE

To investigate occupational risk factors for bladder cancer in seven Canadian provinces.

METHODS

We analysed a population-based case-control dataset of 887 individuals with incident, histologically confirmed bladder cancer between 1994 and 1997. Controls (2847) frequency matched for age and gender were surveyed in 1996. Questionnaires were returned by about 60% of subjects. Odds ratios (ORs) for occupations and self-reported exposures were adjusted for province, age, race, smoking, and several dietary factors, using unconditional logistic regression.

RESULTS

Statistically significant increased risks were observed among men employed as hairdressers (OR = 3.42; 1.09-10.8), primary metal workers (OR = 2.40; 1.29-4.50), miners (OR = 1.94; 1.18-3.17), and automechanics (OR = 1.69; 1.02-2.82). Primary metal workers and automechanics showed evidence of an employment duration-response trend. Modest elevated risks that were not significant were also observed for male government inspectors, printers, firefighters, general labourers, and welders. A duration-response trend was evident for government inspectors and general labourers. For females, significant elevations were observed among lumber processors (OR = 8.78; 1.28-60.1), general labourers (OR = 2.18; 1.05-4.52), nurses (OR = 1.54; 1.03-2.31), and general clerks (OR = 1.48; 1.01-2.17). The latter showed a positive duration-response trend.

CONCLUSIONS

This study found a statistically significant excess risk of bladder cancer, with a duration-response trend, among male primary metal workers and automechanics, and female office workers engaged in general clerical duties.

A Randomized Trial of Incentives to Improve Response Rates to a Mailed Women's Health Questionnaire

BACKGROUND

Mailed questionnaires can be a convenient method for collecting data on women's health, although poor response rates are a concern.

METHODS

As part of a survey of women's health conducted in Maryland in 2001, a randomized trial was performed to assess the effects of two incentives (US dollars 1.00 or a lottery ticket) as well as precontact with an introductory postcard on response rates. Questionnaires were mailed to 3000 women aged 40-60 who were randomized to one of six incentive/precontact groups: lottery/postcard, money/postcard, postcard only, lottery only, money only, or no incentive/no postcard.

RESULTS

The overall response rate was 37.6%. Each incentive/precontact group yielded a higher response rate than the no incentive/no postcard group, although only the response rates for the lottery/postcard group (41.3%) and the money only group (40.0%) were significantly higher than that of the no incentive/no postcard group (33.1%). Money was the only factor that had a significant independent effect on likelihood of response (hazards ratio [HR] compared to no incentive = 1.22, 95% confidence interval [CI] = 1.03, 1.43). Response rates were lower in minority ZIP codes, although the effects of the incentives were generally greater than in the nonminority ZIP codes.

CONCLUSIONS

These results indicate that response rates to mailed women's health questionnaires may be improved with modest incentives, particularly cash incentives.

A survey of non-accidental injury imaging in England, Scotland and Wales

AIM

To identify the potential national variation in non-accidental injury (NAI) imaging in England, Scotland and Wales.

MATERIALS AND METHODS

A postal survey was sent to 323 hospitals with both paediatric and radiology departments. These were identified by a search through the Medical Directory.

RESULTS

One hundred and thirteen of 323 postal questionnaires were returned within the study period (35%). Sixteen were excluded from the study because either no NAI imaging was performed at that institution or an incorrect address had been used. The total number of completed questionnaires was 97 (30%). Extensive variation was seen in all aspects of NAI imaging including imaging techniques used, total case numbers, follow-up imaging and those who report the NAI imaging.

CONCLUSIONS

There is currently no national protocol that incorporates all aspects of NAI imaging in England, Scotland and Wales. Extensive variation in practice has been shown by this survey. Further standardization of NAI imaging practice is required. The draft BSPR skeletal survey guidelines and routine neurological imaging is recommended.

Using commercial telephone directories to obtain a population-based sample for mail survey of women of reproductive age

In the United States, sampling women of reproductive age from the general population for research purposes is a challenge. Even more difficult is conducting a population-based study of couples attempting pregnancy to assess fecundity and fertility or related impairments. To address the problem of obtaining representative samples from the population in order to study such health-related issues, a commercially and readily available CD-ROM telephone directory was used and tested as a sampling framework for studies aimed at enrolling gravid women aged 18-44 years. A self-administered questionnaire (SAQ) was mailed to a stratified random sample of 10 005 (3%) households in Erie County, NY, USA. Overall, 17% of the questionnaires were undeliverable despite updating all addresses with residential software before mailing. Thirteen per cent (n = 1089) of the households returned completed questionnaires, of which 35% (n = 377) were completed by women aged 18-44 years. Using 1990 census information for zip code, respondents were more likely to be white and to have higher median household incomes than non-respondents. Of the 377 women who completed the questionnaire, 79% had been pregnant at least once, 5% reported being unable to become pregnant, and 16% reporting never trying to become pregnant. Despite the overall low response to the SAQ, the sampling framework captured a diverse group of women of reproductive age who reported various fecundity and fertility outcomes. The use of low-cost commercially available software linked to census data for selecting samples of women or couples for reproductive and perinatal research may be possible; however, oversampling of households, use of incentives and follow-up of non-respondents is needed to ensure adequate sample sizes.