Impact of Incentives on Physician Participation in Research Surveys: Randomized Experiment

BACKGROUND

Web-based surveys can be effective data collection instruments; however, participation is notoriously low, particularly among professionals such as physicians. Few studies have explored the impact of varying amounts of monetary incentives on survey completion.

OBJECTIVE

This study aims to conduct a randomized study to assess how different incentive amounts influenced survey participation among neurologists in the United States.

METHODS

We distributed a web-based survey using standardized email text to 21,753 individuals randomly divided into 5 equal groups (≈4351 per group). In phase 1, each group was assigned to receive either nothing or a gift card for US $10, $20, $50, or $75, which was noted in the email subject and text. After 4 reminders, phase 2 began and each remaining individual was offered a US $75 gift card to complete the survey. We calculated and compared the proportions who completed the survey by phase 1 arm, both before and after the incentive change, using a chi-square test. As a secondary outcome, we also looked at survey participation as opposed to completion.

RESULTS

For the 20,820 emails delivered, 879 (4.2%) recipients completed the survey; of the 879 recipients, 622 (70.8%) were neurologists. Among the neurologists, most were male (412/622, 66.2%), White (430/622, 69.1%), non-Hispanic (592/622, 95.2%), graduates of American medical schools (465/622, 74.8%), and board certified (598/622, 96.1%). A total of 39.7% (247/622) completed their neurology residency more than 20 years ago, and 62.4% (388/622) practiced in an urban setting. For phase 1, the proportions of respondents completing the survey increased as the incentive amount increased (46/4185, 1.1%; 76/4165, 1.8%; 86/4160, 2.1%; 104/4162, 2.5%; and 119/4148, 2.9%, for US $0, $10, $20, $50, and $75, respectively; P<.001). In phase 2, the survey completion rate for the former US $0 arm increased to 3% (116/3928). Those originally offered US $10, $20, $50, and $75 who had not yet participated were less likely to participate compared with the former US $0 arm (116/3928, 3%; 90/3936, 2.3%; 80/3902, 2.1%; 88/3845, 2.3%; and 74/3878, 1.9%, for US $0, $10, $20, $50, and $75, respectively; P=.03). For our secondary outcome of survey participation, a trend similar to that of survey completion was observed in phase 1 (55/4185, 1.3%; 85/4165, 2%; 96/4160, 2.3%; 118/4162, 2.8%; and 135/4148, 3.3%, for US $0, $10, $20, $50, and $75, respectively; P<.001) and phase 2 (116/3928, 3%; 90/3936, 2.3%; 80/3902, 2.1%; 88/3845, 2.3%; and 86/3845, 2.2%, for US $0, $10, $20, $50, and $75, respectively; P=.10).

CONCLUSIONS

As expected, monetary incentives can boost physician survey participation and completion, with a positive correlation between the amount offered and participation.

Critical Appraisal of Electronic Surveys: An Integrated Literature Review

Background and Purpose: Electronic surveys are an essential data collection method in survey research but there are pros and cons. The purpose of this literature review was to understand the advantages and disadvantages of electronic surveys in research. Methods: An integrated literature review was performed. Results: Twenty publications met the criteria and were analyzed. The advantages of electronic surveys include speed, cost, convenience, flexibility, ease of analyses, global reach, reduced errors, and question diversity. The disadvantages of electronic surveys are response outcomes (nonresponse, item-nonresponse/poor completion rates, and careless responding errors) and digital literacy requirements. Conclusions: The advantages of electronic surveys outweigh their disadvantages, but researchers must understand the problems associated with electronic surveys and avoid them.

An open repository of real-time COVID-19 indicators

To study the COVID-19 pandemic, its effects on society, and measures for reducing its spread, researchers need detailed data on the course of the pandemic. Standard public health data streams suffer inconsistent reporting and frequent, unexpected revisions. They also miss other aspects of a population’s behavior that are worthy of consideration. We present an open database of COVID signals in the United States, measured at the county level and updated daily. This includes traditionally reported COVID cases and deaths, and many others: measures of mobility, social distancing, internet search trends, self-reported symptoms, and patterns of COVID-related activity in deidentified medical insurance claims. The database provides all signals in a common, easy-to-use format, empowering both public health research and operational decision-making.

The COVID-19 pandemic presented enormous data challenges in the United States. Policy makers, epidemiological modelers, and health researchers all require up-to-date data on the pandemic and relevant public behavior, ideally at fine spatial and temporal resolution. The COVIDcast API is our attempt to fill this need: Operational since April 2020, it provides open access to both traditional public health surveillance signals (cases, deaths, and hospitalizations) and many auxiliary indicators of COVID-19 activity, such as signals extracted from deidentified medical claims data, massive online surveys, cell phone mobility data, and internet search trends. These are available at a fine geographic resolution (mostly at the county level) and are updated daily. The COVIDcast API also tracks all revisions to historical data, allowing modelers to account for the frequent revisions and backfill that are common for many public health data sources. All of the data are available in a common format through the API and accompanying R and Python software packages. This paper describes the data sources and signals, and provides examples demonstrating that the auxiliary signals in the COVIDcast API present information relevant to tracking COVID activity, augmenting traditional public health reporting and empowering research and decision-making.

The effect of administration mode on CAHPS survey response rates and results: A comparison of mail and web-based approaches

OBJECTIVE

The objective of this study was to compare response rates, respondents' characteristics, and substantive results for CAHPS surveys administered using web and mail protocols.

DATA SOURCES

Patients who had one or more primary care visits in the preceding 6 months.

STUDY DESIGN/DATA COLLECTION METHODS

Patients for whom primary care practices had email addresses were randomized to one of four survey administration protocols: web via a portal invitation; web via an email invitation; combination of web and mail; and mail only. Another sample of patients without known email addresses was surveyed by mail. Samples of nonrespondents to the Internet and mail protocols were surveyed by telephone.

PRINCIPAL FINDINGS

Response rates to surveys administered using the Internet protocols were lower than for the surveys administered by mail (20 percent vs over 40 percent). However, characteristics of respondents and survey answers were very similar across protocols. Respondents without email addresses were older, less educated, and more likely to be male than those with email addresses, and there were a few differences in their responses. There was little evidence of nonresponse bias in either the mail or web protocols.

CONCLUSION

In this well-educated patient population, web protocols had lower response rates, but substantive results very similar to those from mail protocols.

Survey mode effects on valuation of environmental goods

This article evaluates the effect of the choice of survey recruitment mode on the value of water quality in lakes, rivers, and streams. Four different modes are compared: bringing respondents to one central location after phone recruitment, mall intercepts in two states, national phone-mail survey, and an Internet survey with a national, probability-based panel. The modes differ in terms of the representativeness of the samples, non-response rates, sample selection effects, and consistency of responses. The article also shows that the estimated value of water quality can differ substantially depending on the survey mode. The national Internet panel has the most desirable properties with respect to performance on the four important survey dimensions of interest.