Effects of disseminating research findings on response rates in a community survey: a randomised controlled trial

Methods to increase response to postal and electronic questionnaires

BACKGROUND

Self-administered questionnaires are widely used to collect data in epidemiological research, but non-response reduces the effective sample size and can introduce bias. Finding ways to increase response to postal and electronic questionnaires would improve the quality of epidemiological research.

OBJECTIVES

To identify effective strategies to increase response to postal and electronic questionnaires.

SEARCH METHODS

We searched 14 electronic databases up to December 2021 and manually searched the reference lists of relevant trials and reviews. We contacted the authors of all trials or reviews to ask about unpublished trials; where necessary, we also contacted authors to confirm the methods of allocation used and to clarify results presented.

SELECTION CRITERIA

Randomised trials of methods to increase response to postal or electronic questionnaires. We assessed the eligibility of each trial using pre-defined criteria.

DATA COLLECTION AND ANALYSIS

We extracted data on the trial participants, the intervention, the number randomised to intervention and comparison groups and allocation concealment. For each strategy, we estimated pooled odds ratios (OR) and 95% confidence intervals (CI) in a random-effects model. We assessed evidence for selection bias using Egger's weighted regression method and Begg's rank correlation test and funnel plot. We assessed heterogeneity amongst trial odds ratios using a Chi2 test and quantified the degree of inconsistency between trial results using the I2 statistic.

MAIN RESULTS

Postal We found 670 eligible trials that evaluated over 100 different strategies of increasing response to postal questionnaires. We found substantial heterogeneity amongst trial results in half of the strategies. The odds of response almost doubled when: using monetary incentives (odds ratio (OR) 1.86; 95% confidence interval (CI) 1.73 to 1.99; heterogeneity I2 = 85%); using a telephone reminder (OR 1.96; 95% CI 1.03 to 3.74); and when clinical outcome questions were placed last (OR 2.05; 95% CI 1.00 to 4.24). The odds of response increased by about half when: using a shorter questionnaire (OR 1.58; 95% CI 1.40 to 1.78); contacting participants before sending questionnaires (OR 1.36; 95% CI 1.23 to 1.51; I2 = 87%); incentives were given with questionnaires (i.e. unconditional) rather than when given only after participants had returned their questionnaire (i.e. conditional on response) (OR 1.53; 95% CI 1.35 to 1.74); using personalised SMS reminders (OR 1.53; 95% CI 0.97 to 2.42); using a special (recorded) delivery service (OR 1.68; 95% CI 1.36 to 2.08; I2 = 87%); using electronic reminders (OR 1.60; 95% CI 1.10 to 2.33); using intensive follow-up (OR 1.69; 95% CI 0.93 to 3.06); using a more interesting/salient questionnaire (OR 1.73; 95% CI 1.12 to 2.66); and when mentioning an obligation to respond (OR 1.61; 95% CI 1.16 to 2.22). The odds of response also increased with: non-monetary incentives (OR 1.16; 95% CI 1.11 to 1.21; I2 = 80%); a larger monetary incentive (OR 1.24; 95% CI 1.15 to 1.33); a larger non-monetary incentive (OR 1.15; 95% CI 1.00 to 1.33); when a pen was included (OR 1.44; 95% CI 1.38 to 1.50); using personalised materials (OR 1.15; 95% CI 1.09 to 1.21; I2 = 57%); using a single-sided rather than a double-sided questionnaire (OR 1.13; 95% CI 1.02 to 1.25); using stamped return envelopes rather than franked return envelopes (OR 1.23; 95% CI 1.13 to 1.33; I2 = 69%), assuring confidentiality (OR 1.33; 95% CI 1.24 to 1.42); using first-class outward mailing (OR 1.11; 95% CI 1.02 to 1.21); and when questionnaires originated from a university (OR 1.32; 95% CI 1.13 to 1.54). The odds of response were reduced when the questionnaire included questions of a sensitive nature (OR 0.94; 95% CI 0.88 to 1.00). Electronic We found 88 eligible trials that evaluated over 30 different ways of increasing response to electronic questionnaires. We found substantial heterogeneity amongst trial results in half of the strategies. The odds of response tripled when: using a brief letter rather than a detailed letter (OR 3.26; 95% CI 1.79 to 5.94); and when a picture was included in an email (OR 3.05; 95% CI 1.84 to 5.06; I2 = 19%). The odds of response almost doubled when: using monetary incentives (OR 1.88; 95% CI 1.31 to 2.71; I2 = 79%); and using a more interesting topic (OR 1.85; 95% CI 1.52 to 2.26). The odds of response increased by half when: using non-monetary incentives (OR 1.60; 95% CI 1.25 to 2.05); using shorter e-questionnaires (OR 1.51; 95% CI 1.06 to 2.16; I2 = 94%); and using a more interesting e-questionnaire (OR 1.85; 95% CI 1.52 to 2.26). The odds of response increased by a third when: offering survey results as an incentive (OR 1.36; 95% CI 1.16 to 1.59); using a white background (OR 1.31; 95% CI 1.10 to 1.56); and when stressing the benefits to society of response (OR 1.38; 95% CI 1.07 to 1.78; I2 = 41%). The odds of response also increased with: personalised e-questionnaires (OR 1.24; 95% CI 1.17 to 1.32; I2 = 41%); using a simple header (OR 1.23; 95% CI 1.03 to 1.48); giving a deadline (OR 1.18; 95% CI 1.03 to 1.34); and by giving a longer time estimate for completion (OR 1.25; 95% CI 0.96 to 1.64). The odds of response were reduced when: "Survey" was mentioned in the e-mail subject (OR 0.81; 95% CI 0.67 to 0.97); when the email or the e-questionnaire was from a male investigator, or it included a male signature (OR 0.55; 95% CI 0.38 to 0.80); and by using university sponsorship (OR 0.84; 95%CI 0.69 to 1.01). The odds of response using a postal questionnaire were over twice those using an e-questionnaire (OR 2.33; 95% CI 2.25 to 2.42; I2 = 98%). Response also increased when: providing a choice of response mode (electronic or postal) rather than electronic only (OR 1.76 95% CI 1.67 to 1.85; I2 = 97%); and when administering the e-questionnaire by computer rather than by smartphone (OR 1.62 95% CI 1.36 to 1.94).

AUTHORS' CONCLUSIONS

Researchers using postal and electronic questionnaires can increase response using the strategies shown to be effective in this Cochrane review.

HBM4EU feasibility studies: Lessons learned in combining health and human biomonitoring studies

BACKGROUND

The European Human Biomonitoring Initiative (HBM4EU) is a joint program evaluating humans' exposure to several environmental substances and their potential health effects. One of the main objectives of HBM4EU is to make use of human biomonitoring (HBM) to assess human exposure to chemicals in Europe to better understand the associated health impacts and to improve chemical risk assessment. In parallel to HBM studies, health examination surveys (HESs), nutrition/dietary surveys, and disease specific health surveys are conducted in many European countries. In HESs, information collected by questionnaire(s) is supplemented with physical examinations and analysis of clinical and biological biomarkers in biological samples. HBM and health examination survey (HES) use similar data collection methods and infrastructures hence the feasibility of combining these two is explored in this paper.

METHODS

Within HBM4EU, three feasibility studies (in Finland, Germany, and UK/England) were conducted to evaluate opportunities and obstacles of combining HBM and health studies. In this paper we report lessons learned from these feasibility studies.

RESULTS

The Finnish feasibility study called KouBio-KUOPIO study was a new initiative without links to existing studies. The German feasibility study added a HBM module to the first follow-up examination of the LIFE-Adult-Study, a population-based cohort study. The UK feasibility integrates a sustainable HBM module into the Health Survey for England (HSfE), an annual health examination survey. Benefits of combining HBM and HESs include the use of shared infrastructures. Furthermore, participants can receive additional health information from HES, and participation rates tend to be higher due to the potential to obtain personal health information. Preparatory phases including obtaining ethical approval can be time-consuming and complicated. Recruitment of participants and low participation rates are common concerns in survey research and therefore designing user-friendly questionnaires with low participant burden is important. Unexpected events such as the COVID-19 pandemic can cause substantial challenges and delays for such studies. Furthermore, experiences from several countries demonstrated that long-term funding for combined studies can be difficult to obtain.

CONCLUSIONS

In the future, incorporating HBM modules into existing HESs can provide a feasible and cost-effective method to conduct HBM studies and obtain a wide range of relevant data to support public health policies and research.

Most response-inducing strategies do not increase participation in observational studies: a systematic review and meta-analysis

OBJECTIVES

To evaluate response-inducing strategies for observational studies using health-related questionnaires or interviews.

STUDY DESIGN AND SETTING

We searched PubMed, EMBASE, CINAHL, PsycINFO, and Web of Science up to December 28, 2017. Studies evaluating the effect of a response-inducing strategy on participation rates of observational studies were included. For each strategy, we estimated pooled response ratios with 95% confidence intervals (CIs) in a Hartung-Knapp/Sidik-Jonkman random effects model with the final participation rate as outcome, stratified for type of participants and method of data collection.

RESULTS

The search yielded 168 eligible studies involving 367,616 potential participants and 33 strategies. Among patients, response-inducing strategies for paper-based questionnaires included unconditional monetary incentives (response ratio 1.15; 95% CI 1.09-1.21) and shorter questionnaires (1.04; 1.02-1.06). Among nonpatients, a personalized mode of delivery (1.47; 1.24-1.74), more expensive mailing type (1.25; 1.00-1.56), unconditional monetary incentives (1.24; 1.12-1.38), prenotification (1.12; 1.03-1.22), unconditional scratch lottery tickets (1.09; 1.01-1.18), and shorter questionnaires (1.06; 1.02-1.11) increased response rates to paper-based questionnaires. For Web-based questionnaires and interviews among nonpatients, response rates were increased by conditional lottery tickets (1.17; 1.02-1.34) and conditional monetary incentives (1.39; 1.01-1.91), respectively.

CONCLUSION

Although the majority of strategies evaluated were unsuccessful, some may increase response rates to observational studies, particularly among nonpatients.

Whither survey research?

In April 2004 new requirements for research governance were introduced in England. In order to safeguard the interests of research participants, research involving NHS patients, their tissues or NHS staff has to be registered with each organisation from where data will be collected and authorisation obtained before the study can commence. NHS Trusts/Primary Care Trusts (PCTs) have set up individual mechanisms for research governance approval with the result that there are no standardised procedures or documentation.

Drawing upon data derived from making research governance applications for a postal survey involving nurses in 90 NHS Trusts/PCTs in seven strategic health authorities in England, this paper examines the implications of introducing research governance for this research method. An analysis of the application process, and the time and resources required suggests that undertaking survey research is increasingly problematic. Moreover, implications arising from the Data Protection Act and the need to complete commissioned research within contractual arrangements led to methodological compromises being made. Recommendations for changes in research governance policy are proposed, most notably the need to standardise procedures, provide greater clarification for R&D managers on the interpretation of research governance procedures and apportion the level of scrutiny according to risk.

Methods to increase response to postal and electronic questionnaires

BACKGROUND

Postal and electronic questionnaires are widely used for data collection in epidemiological studies but non-response reduces the effective sample size and can introduce bias. Finding ways to increase response to postal and electronic questionnaires would improve the quality of health research.

OBJECTIVES

To identify effective strategies to increase response to postal and electronic questionnaires.

SEARCH STRATEGY

We searched 14 electronic databases to February 2008 and manually searched the reference lists of relevant trials and reviews, and all issues of two journals. We contacted the authors of all trials or reviews to ask about unpublished trials. Where necessary, we also contacted authors to confirm methods of allocation used and to clarify results presented. We assessed the eligibility of each trial using pre-defined criteria.

SELECTION CRITERIA

Randomised controlled trials of methods to increase response to postal or electronic questionnaires.

DATA COLLECTION AND ANALYSIS

We extracted data on the trial participants, the intervention, the number randomised to intervention and comparison groups and allocation concealment. For each strategy, we estimated pooled odds ratios (OR) and 95% confidence intervals (CI) in a random-effects model. We assessed evidence for selection bias using Egger's weighted regression method and Begg's rank correlation test and funnel plot. We assessed heterogeneity among trial odds ratios using a Chi(2) test and the degree of inconsistency between trial results was quantified using the I(2) statistic.

MAIN RESULTS

PostalWe found 481 eligible trials. The trials evaluated 110 different ways of increasing response to postal questionnaires. We found substantial heterogeneity among trial results in half of the strategies. The odds of response were at least doubled using monetary incentives (odds ratio 1.87; 95% CI 1.73 to 2.04; heterogeneity P < 0.00001, I(2) = 84%), recorded delivery (1.76; 95% CI 1.43 to 2.18; P = 0.0001, I(2) = 71%), a teaser on the envelope - e.g. a comment suggesting to participants that they may benefit if they open it (3.08; 95% CI 1.27 to 7.44) and a more interesting questionnaire topic (2.00; 95% CI 1.32 to 3.04; P = 0.06, I(2) = 80%). The odds of response were substantially higher with pre-notification (1.45; 95% CI 1.29 to 1.63; P < 0.00001, I(2) = 89%), follow-up contact (1.35; 95% CI 1.18 to 1.55; P < 0.00001, I(2) = 76%), unconditional incentives (1.61; 1.36 to 1.89; P < 0.00001, I(2) = 88%), shorter questionnaires (1.64; 95% CI 1.43 to 1.87; P < 0.00001, I(2) = 91%), providing a second copy of the questionnaire at follow up (1.46; 95% CI 1.13 to 1.90; P < 0.00001, I(2) = 82%), mentioning an obligation to respond (1.61; 95% CI 1.16 to 2.22; P = 0.98, I(2) = 0%) and university sponsorship (1.32; 95% CI 1.13 to 1.54; P < 0.00001, I(2) = 83%). The odds of response were also increased with non-monetary incentives (1.15; 95% CI 1.08 to 1.22; P < 0.00001, I(2) = 79%), personalised questionnaires (1.14; 95% CI 1.07 to 1.22; P < 0.00001, I(2) = 63%), use of hand-written addresses (1.25; 95% CI 1.08 to 1.45; P = 0.32, I(2) = 14%), use of stamped return envelopes as opposed to franked return envelopes (1.24; 95% CI 1.14 to 1.35; P < 0.00001, I(2) = 69%), an assurance of confidentiality (1.33; 95% CI 1.24 to 1.42) and first class outward mailing (1.11; 95% CI 1.02 to 1.21; P = 0.78, I(2) = 0%). The odds of response were reduced when the questionnaire included questions of a sensitive nature (0.94; 95% CI 0.88 to 1.00; P = 0.51, I(2) = 0%).ElectronicWe found 32 eligible trials. The trials evaluated 27 different ways of increasing response to electronic questionnaires. We found substantial heterogeneity among trial results in half of the strategies. The odds of response were increased by more than a half using non-monetary incentives (1.72; 95% CI 1.09 to 2.72; heterogeneity P < 0.00001, I(2) = 95%), shorter e-questionnaires (1.73; 1.40 to 2.13; P = 0.08, I(2) = 68%), including a statement that others had responded (1.52; 95% CI 1.36 to 1.70), and a more interesting topic (1.85; 95% CI 1.52 to 2.26). The odds of response increased by a third using a lottery with immediate notification of results (1.37; 95% CI 1.13 to 1.65), an offer of survey results (1.36; 95% CI 1.15 to 1.61), and using a white background (1.31; 95% CI 1.10 to 1.56). The odds of response were also increased with personalised e-questionnaires (1.24; 95% CI 1.17 to 1.32; P = 0.07, I(2) = 41%), using a simple header (1.23; 95% CI 1.03 to 1.48), using textual representation of response categories (1.19; 95% CI 1.05 to 1.36), and giving a deadline (1.18; 95% CI 1.03 to 1.34). The odds of response tripled when a picture was included in an e-mail (3.05; 95% CI 1.84 to 5.06; P = 0.27, I(2) = 19%). The odds of response were reduced when "Survey" was mentioned in the e-mail subject line (0.81; 95% CI 0.67 to 0.97; P = 0.33, I(2) = 0%), and when the e-mail included a male signature (0.55; 95% CI 0.38 to 0.80; P = 0.96, I(2) = 0%).

AUTHORS' CONCLUSIONS

Health researchers using postal and electronic questionnaires can increase response using the strategies shown to be effective in this systematic review.

Use of a Local Worker Survey as a Source of Neighborhood Information

Researchers increasingly recognize the potential influence of the neighborhood environment on individual health and social behavior. To examine these influences, it is important to use varying measures and sources of neighborhood characteristics. Though neighborhood residents are often surveyed, the perceptions of neighborhood workers have been largely ignored. The current study documents procedures and findings from two longitudinal studies in which workers in 60 neighborhoods were surveyed about neighborhood social cohesion and social control (collective efficacy), and neighborhood problems. Results indicated that workers within neighborhoods were more homogeneous in their views of neighborhood collective efficacy and neighborhood problems than were workers across neighborhoods. In addition, workers' perceptions of their neighborhoods were similar to the perceptions of neighborhood residents, but also provided unique information. Overall, this study demonstrates the viability and usefulness of local workers as an additional source of neighborhood information.

Predictors of complementary therapy use among asthma patients: results of a primary care survey

Patients with chronic conditions are increasingly using complementary therapies. Asthma is the most common chronic disease in the UK. Qualitative research has suggested reasons why asthma patients use complementary therapies. However, there is little reliable quantitative evidence regarding the prevalence of complementary therapy use among asthma patients and predictors of use. A postal survey of complementary therapy use among asthma patients was therefore conducted via 27 general practices across seven Primary Care Trusts within the South West Strategic Health Authority (England), during August 2005 to May 2006. A total of 14,833 asthma patients were identified. A 1-in-4 random sample generated 3693 potential respondents, of whom 1320 (36%) returned questionnaires. Taking full account of the survey design, 14.5% (190/1308; 95% confidence interval 12.5% to 16.6%) had used complementary therapies for asthma; 54% of these patients had not disclosed their complementary therapy use to a health professional. The three therapies most commonly used were homeopathy, herbal medicine and relaxation. Just over half of those using complementary therapies for asthma reported that they usually or always helped; the most common reported benefits were symptom reduction, calming breathing and reducing panic. Multivariable analyses indicated an inverted U-shaped relationship between complementary therapy use for asthma and age, and increased likelihood of use among women, those with educational qualifications, those not usually helped by asthma medication, and those who have difficulty sleeping because of asthma symptoms. Dissatisfaction with conventional care was not associated with complementary therapy use for asthma. Asthma patients may use complementary therapies with or without the knowledge of their healthcare providers. Open communication between professionals and patients about complementary therapies may be valuable to give patients enhanced opportunities to discuss the impact of asthma on their quality of life and the effectiveness of their conventional treatment.