Association between response rates and monetary incentives in sample study: a systematic review and meta-analysis

The Impact of Text Messaging to Promote Patient Reported Outcome Measures (PROMs) Completion in Orthopedic Practice: Findings From a Randomized Controlled Study

It is difficult to achieve high response rates to Patient Reported Outcome Measures (PROMs) surveys collected as part of clinical care. However, they are operationally and clinically important. To understand the impact of text message reminders on response rates to PROMs collected via email as part of routine care for hip or knee replacement surgery, initial nonresponders were randomized to receive a text reminder or not at 7 and 12 days, if needed. At day 7, the overall survey response rate was 63%. Model-derived estimates for survey return after this point were 51.1% (95% confidence interval [CI], 48.0%-54.2%) in the text arm compared to 34.5% (95% CI, 31.6%-37.6%) in the no text arm. The effect of text messages on response rates did not vary by subpopulations considered nor were there differences in rates of key outcomes between the 2 groups, suggesting that it did not impact any underlying response bias. Given the relative low cost of text messages, they can be an efficient means to increase response rates.

Testing a decoy donation incentive to improve online survey participation: Evidence from a field experiment

This study introduces a new randomized field experiment exploring the impact of offering a decoy charity donation incentive together with a monetary reward to increase response rates in an online survey about coronavirus fears. The study used a two-stage approach, starting with a preliminary survey to investigate participant attitudes toward different types of donations. Subsequently, an experiment was conducted wherein a less desirable £2 donation (the decoy) was introduced as an alternative to a £2 Amazon voucher (the target) within the choice set. The study sample consisted of 431 university students. They were split into three groups: a control group with a standard £2 Amazon voucher incentive (216 participants), a decoy group with the target shown first (108 participants), and a decoy group with the decoy shown first (107 participants). We found significantly higher survey completion rates in the decoy than in the control condition (82.3% vs. 74.5%). Notably, an order effect was observed-presenting the target before the decoy led to a higher completion rate (89.8%) compared to presenting the decoy first (74.8%). Importantly, the inclusion of the decoy incentive did not introduce any response bias. This study offers a proof of principle that incorporating a decoy charity donation incentive into the choice set can have a positive impact on survey participation without adversely affecting response behaviour. It demonstrates the potential of such incentives to encourage participants to complete online surveys, even when a small monetary reward is offered.

Adaptations for remote research work: a modified web-push strategy compared to a mail-only strategy for administering a survey of healthcare experiences

BACKGROUND

The COVID-19 pandemic required that our research team change our mail-only (MO) strategy for a research survey to a strategy more manageable by staff working remotely. We used a modified web-push approach (MWP), in which patients were mailed a request to respond online and invited to call if they preferred the questionnaire by mail or phone. We also changed from a pre-completion gift to a post-completion gift card incentive. Our objective is to compare response patterns between modes for a survey that used an MO strategy pre-pandemic followed by an MWP strategy peri-pandemic for data collection.

METHODS

Observational study using data from a national multi-scale survey about patients' experience of specialty care coordination administered via MO in 2019 and MWP from 2020 to 2021 to Veterans receiving primary care and specialty care within the Veterans Health Administration (VA). We compared response rates, respondent characteristics and responses about care coordination between MO and MWP, applying propensity weights to account for differences in the underlying samples.

RESULTS

The response rate was lower for MWP vs. MO (13.4% vs. 36.6%), OR = 0.27, 95% CI = 0.25-0.30, P < .001). Respondent characteristics were similar across MO and MWP. Coordination scale scores tended to be slightly higher for MWP, but the effect sizes for these differences between modes were small for 9 out of 10 scales.

CONCLUSIONS

While the logistics of MWP survey data collection are well-suited to the remote research work environment, response rates were lower than those for the MO method. Future studies should examine addition of multi-mode contacts and/or pre-completion incentives to increase response rates for MWP.

Six ways to maximize survey response rates: lessons from a medical school accreditation survey in a Canadian setting

Background

Surveys are being increasingly used to gather feedback and study data in healthcare professions. However, it may be challenging to achieve high response rates in surveys administered to healthcare professionals. The aim of this paper is to report six strategies that contributed to a high response rate on the Independent Student Analysis at the University of Toronto (U of T), which can be applied to other surveys to achieve strong response rates amongst healthcare professionals.

Methods

In 2019, as part of accreditation for the U of T MD Program, we conducted the Independent Student Analysis, a student-led survey examining a medical student's experience. We review and critically evaluate the factors that contributed to a robust response rate amongst one of the largest cohorts of medical students in Canada.

Results

Among 1080 students in the MD program, we achieved an unprecedented response rate of 87.2%. Six factors were identified that most contributed to our high response rate, including: faculty support, student representation, eliciting participant feedback, creating protected time for completion, offering incentives, and generating awareness.

Conclusions

Eliciting high survey response rates from medical learners can be challenging. However, with careful consideration of learner feedback and effective employment of the strategies discussed in this paper, medical school faculty may better engage students in survey completion, achieving higher response rates and gathering richer insight, which can be used to more effectively enact meaningful change amongst healthcare professionals.

Strategies for Recruiting Migrants to Participate in a Sexual Health Survey: Methods, Results, and Lessons

In this article, we describe the approaches taken to recruit adult migrants living in Australia for a sexual health and blood-borne virus survey (paper and online) and present data detailing the outcomes of these approaches. The purpose was to offer guidance to redress the under-representation of migrants in public health research. Methods of recruitment included directly contacting people in individual/organizational networks, social media posts/advertising, promotion on websites, and face-to-face recruitment at public events/venues. Search query strings were used to provide information about an online referral source, and project officers kept records of activities and outcomes. Descriptive statistical analyses were used to determine respondent demographic characteristics, proportions recruited to complete the paper and online surveys, and sources of referral. Logistic regression analyses were run to predict online participation according to demographic characteristics. The total sample comprised 1454 African and Asian migrants, with 59% identifying as female. Most respondents (72%) were recruited to complete the paper version of the survey. Face-to-face invitations resulted in the highest number of completions. Facebook advertising did not recruit large numbers of respondents. Same-sex attraction and age (40-49 years) were statistically significant predictors of online completion. We encourage more researchers to build the evidence base on ways to produce research that reflects the needs and perspectives of minority populations who often bear the greatest burden of disease.

How Are Albertans "Adjusting to and Coping With" Dialysis? A Cross-Sectional Survey

Background:

Depression and anxiety are commonly reported (40% and 11%-52%) among adults receiving dialysis, compared with ~10% among all Canadians. Mental health in dialysis care is underrecognized and undertreated.

Objective:

(1) To describe preferences for mental health support reported by Albertans receiving dialysis; (2) to compare depression, anxiety, and quality-of-life (QOL) domains for people who would or would not engage in support for mental health; and (3) to explore sociodemographic, mental health, and QOL domains that explain whether people would or would not engage in support for mental health.

Design:

A cross-sectional survey.

Setting:

Alberta, Canada.

Patients:

Adults receiving all modalities of dialysis (N = 2972).

Measurements:

An online survey with questions about preferences for mental health support and patient-reported outcome measures (Patient Health Questionnaire–9 [PHQ-9], Generalized Anxiety Disorder–7 [GAD-7], and Kidney Disease QOL Instrument–36 [KDQOL-36]).

Methods:

To address objectives 1 and 2, we conducted chi-square tests (for discrete variables) and t tests (for continuous variables) to compare the distributions of the above measures for two groups: Albertans receiving dialysis who would engage or would not engage in support for mental health. We subsequently conducted a series of binary logistic regressions guided by the purposeful variable selection approach to identify a subset of the most relevant explanatory variables for determining whether or not people are more likely to engage in support for mental health (objective 3). To further explain differences between the two groups, we analyzed open-text comments following a summative content analysis approach.

Results:

Among 384 respondents, 72 did not provide a dialysis modality or answer the PHQ-9. The final data set included responses from 312 participants. Of these, 59.6% would consider engaging in support, including discussing medication with a family doctor (72.1%) or nephrologist (62.9%), peer support groups (64.9%), and talk therapy (60%). Phone was slightly favored (73%) over in person at dialysis (67.6%), outpatient (67.2%), or video (59.4%). Moderate to severe depressive symptoms (PHQ-9 score ≥10) was reported by 33.4%, and most respondents (63.9%) reported minimal anxiety symptoms; 36.1% reported mild to severe anxiety symptoms (GAD-7 score ≥5). The mean (SD) PHQ-9 score was 8.9 (6.4) for those who would engage in support, and lower at 5.8 (4.8) for those who would not. The mean (SD) GAD-7 score was 5.2 (5.6) for those who would engage in support and 2.8 (4.1) for those who would not. In the final logistic regression model, people who were unable to work had 2 times the odds of engaging in support than people who are able to work. People were also more likely to engage in support if they had been on dialysis for fewer years and had lower (worse) mental health scores (odds ratios = 1.06 and 1.38, respectively). The final model explained 15.5% (Nagelkerke R²) of the variance and with 66.6% correct classification. We analyzed 146 comments in response to the question, “Is there anything else you like to tell us.” The top 2 categories for both groups were QOL and impact of dialysis environment. The third category differed: those who would engage wrote about support, whereas those who would not engage wrote about “dialysis is the least of my worries.”

Limitations:

A low response rate of 12.9% limits representativeness; people who chose not to participate may have different experiences of mental health.

Conclusions:

Incorporating patients’ preferences and willingness to engage in support for mental health will inform future visioning for person-centered mental health care in dialysis.

Perception of clinical research among patients and healthy volunteers of clinical trials

Purpose

Clinical research relies on data from patients and volunteers, yet the target sample size is often not achieved. Here, we assessed the perception of clinical research among clinical trial participants to improve the recruitment process for future studies.

Methods

We conducted a single-center descriptive and exploratory study of 300 current or former participants in various phase I–III clinical trials. Questionnaires were either distributed to current clinical trial participants or emailed to former subjects.

Results

Subjects strongly agreed or agreed that contributing to improving medical care (> 81%), contributing to scientific research (> 79%), and trusting their treating physicians (> 77%) were motives for study participation. Among healthy volunteers, financial motives positively correlated with the number of clinical trials they had participated in (p < 0.05). Higher age positively correlated with expectation of best available treatment during study participation among patients (p < 0.05). Less than 8% of all subjects expressed “great concern” about the potential risks of sharing their personal information as part of the study. Subjects displayed “great trust” or “trust” in medical staff (86.6%) and in government research institutions (76.4%), and “very little trust” or “little trust” in pharmaceutical companies (35.4%) and health insurance companies (16.9%).

Conclusion

Altruistic motives and trust in treating physicians were predominant motives for clinical trial participation. Older patients expected to receive the best available treatment during participation. Healthy volunteers who reported financial motives had participated in more clinical trials. Consistent with great trust in medical staff and government research institutions, little concern was expressed about the misuse of personal data during the trial.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00228-022-03366-3.

Improving Electronic Survey Response Rates Among Cancer Center Patients During the COVID-19 Pandemic: Mixed Methods Pilot Study

BACKGROUND

Surveys play a vital role in cancer research. During the COVID-19 pandemic, the use of electronic surveys is crucial to improve understanding of the patient experience. However, response rates to electronic surveys are often lower compared with those of paper surveys.

OBJECTIVE

The aim of this study was to determine the best approach to improve response rates for an electronic survey administered to patients at a cancer center during the COVID-19 pandemic.

METHODS

We contacted 2750 patients seen at Moffitt Cancer Center in the prior 5 years via email to complete a survey regarding their experience during the COVID-19 pandemic, with patients randomly assigned to a series of variations of prenotifications (ie, postcard, letter) or incentives (ie, small gift, modest gift card). In total, eight combinations were evaluated. Qualitative interviews were conducted to understand the level of patient understanding and burden with the survey, and quantitative analysis was used to evaluate the response rates between conditions.

RESULTS

A total of 262 (9.5%) patients completed the survey and 9 participated in a qualitative interview. Interviews revealed minimal barriers in understanding or burden, which resulted in minor survey design changes. Compared to sending an email only, sending a postcard or letter prior to the email improved response rates from 3.7% to 9.8%. Similarly, inclusion of an incentive significantly increased the response rate from 5.4% to 16.7%, especially among racial (3.0% to 12.2%) and ethnic (6.4% to 21.0%) minorities, as well as among patients with low socioeconomic status (3.1% to 14.9%).

CONCLUSIONS

Strategies to promote effective response rates include prenotification postcards or letters as well as monetary incentives. This work can inform future survey development to increase response rates for electronic surveys, particularly among hard-to-reach populations.