The effects of continuing medical education credits on physician response rates to a mailed questionnaire

Methods to increase response to postal and electronic questionnaires

BACKGROUND

Self-administered questionnaires are widely used to collect data in epidemiological research, but non-response reduces the effective sample size and can introduce bias. Finding ways to increase response to postal and electronic questionnaires would improve the quality of epidemiological research.

OBJECTIVES

To identify effective strategies to increase response to postal and electronic questionnaires.

SEARCH METHODS

We searched 14 electronic databases up to December 2021 and manually searched the reference lists of relevant trials and reviews. We contacted the authors of all trials or reviews to ask about unpublished trials; where necessary, we also contacted authors to confirm the methods of allocation used and to clarify results presented.

SELECTION CRITERIA

Randomised trials of methods to increase response to postal or electronic questionnaires. We assessed the eligibility of each trial using pre-defined criteria.

DATA COLLECTION AND ANALYSIS

We extracted data on the trial participants, the intervention, the number randomised to intervention and comparison groups and allocation concealment. For each strategy, we estimated pooled odds ratios (OR) and 95% confidence intervals (CI) in a random-effects model. We assessed evidence for selection bias using Egger's weighted regression method and Begg's rank correlation test and funnel plot. We assessed heterogeneity amongst trial odds ratios using a Chi2 test and quantified the degree of inconsistency between trial results using the I2 statistic.

MAIN RESULTS

Postal We found 670 eligible trials that evaluated over 100 different strategies of increasing response to postal questionnaires. We found substantial heterogeneity amongst trial results in half of the strategies. The odds of response almost doubled when: using monetary incentives (odds ratio (OR) 1.86; 95% confidence interval (CI) 1.73 to 1.99; heterogeneity I2 = 85%); using a telephone reminder (OR 1.96; 95% CI 1.03 to 3.74); and when clinical outcome questions were placed last (OR 2.05; 95% CI 1.00 to 4.24). The odds of response increased by about half when: using a shorter questionnaire (OR 1.58; 95% CI 1.40 to 1.78); contacting participants before sending questionnaires (OR 1.36; 95% CI 1.23 to 1.51; I2 = 87%); incentives were given with questionnaires (i.e. unconditional) rather than when given only after participants had returned their questionnaire (i.e. conditional on response) (OR 1.53; 95% CI 1.35 to 1.74); using personalised SMS reminders (OR 1.53; 95% CI 0.97 to 2.42); using a special (recorded) delivery service (OR 1.68; 95% CI 1.36 to 2.08; I2 = 87%); using electronic reminders (OR 1.60; 95% CI 1.10 to 2.33); using intensive follow-up (OR 1.69; 95% CI 0.93 to 3.06); using a more interesting/salient questionnaire (OR 1.73; 95% CI 1.12 to 2.66); and when mentioning an obligation to respond (OR 1.61; 95% CI 1.16 to 2.22). The odds of response also increased with: non-monetary incentives (OR 1.16; 95% CI 1.11 to 1.21; I2 = 80%); a larger monetary incentive (OR 1.24; 95% CI 1.15 to 1.33); a larger non-monetary incentive (OR 1.15; 95% CI 1.00 to 1.33); when a pen was included (OR 1.44; 95% CI 1.38 to 1.50); using personalised materials (OR 1.15; 95% CI 1.09 to 1.21; I2 = 57%); using a single-sided rather than a double-sided questionnaire (OR 1.13; 95% CI 1.02 to 1.25); using stamped return envelopes rather than franked return envelopes (OR 1.23; 95% CI 1.13 to 1.33; I2 = 69%), assuring confidentiality (OR 1.33; 95% CI 1.24 to 1.42); using first-class outward mailing (OR 1.11; 95% CI 1.02 to 1.21); and when questionnaires originated from a university (OR 1.32; 95% CI 1.13 to 1.54). The odds of response were reduced when the questionnaire included questions of a sensitive nature (OR 0.94; 95% CI 0.88 to 1.00). Electronic We found 88 eligible trials that evaluated over 30 different ways of increasing response to electronic questionnaires. We found substantial heterogeneity amongst trial results in half of the strategies. The odds of response tripled when: using a brief letter rather than a detailed letter (OR 3.26; 95% CI 1.79 to 5.94); and when a picture was included in an email (OR 3.05; 95% CI 1.84 to 5.06; I2 = 19%). The odds of response almost doubled when: using monetary incentives (OR 1.88; 95% CI 1.31 to 2.71; I2 = 79%); and using a more interesting topic (OR 1.85; 95% CI 1.52 to 2.26). The odds of response increased by half when: using non-monetary incentives (OR 1.60; 95% CI 1.25 to 2.05); using shorter e-questionnaires (OR 1.51; 95% CI 1.06 to 2.16; I2 = 94%); and using a more interesting e-questionnaire (OR 1.85; 95% CI 1.52 to 2.26). The odds of response increased by a third when: offering survey results as an incentive (OR 1.36; 95% CI 1.16 to 1.59); using a white background (OR 1.31; 95% CI 1.10 to 1.56); and when stressing the benefits to society of response (OR 1.38; 95% CI 1.07 to 1.78; I2 = 41%). The odds of response also increased with: personalised e-questionnaires (OR 1.24; 95% CI 1.17 to 1.32; I2 = 41%); using a simple header (OR 1.23; 95% CI 1.03 to 1.48); giving a deadline (OR 1.18; 95% CI 1.03 to 1.34); and by giving a longer time estimate for completion (OR 1.25; 95% CI 0.96 to 1.64). The odds of response were reduced when: "Survey" was mentioned in the e-mail subject (OR 0.81; 95% CI 0.67 to 0.97); when the email or the e-questionnaire was from a male investigator, or it included a male signature (OR 0.55; 95% CI 0.38 to 0.80); and by using university sponsorship (OR 0.84; 95%CI 0.69 to 1.01). The odds of response using a postal questionnaire were over twice those using an e-questionnaire (OR 2.33; 95% CI 2.25 to 2.42; I2 = 98%). Response also increased when: providing a choice of response mode (electronic or postal) rather than electronic only (OR 1.76 95% CI 1.67 to 1.85; I2 = 97%); and when administering the e-questionnaire by computer rather than by smartphone (OR 1.62 95% CI 1.36 to 1.94).

AUTHORS' CONCLUSIONS

Researchers using postal and electronic questionnaires can increase response using the strategies shown to be effective in this Cochrane review.

An Introductory Guide to Survey Research in Anaesthesia

Surveys allow convenient and inexpensive research. Surveys include mail-out questionnaires, email questionnaires, telephone interviews, and personal interviews. Despite a widespread perception that surveys are easy to conduct, good surveys need rigorous design, implementation and analysis. This requires substantial planning, time and effort. The most important step in designing a survey is to clearly define the question (s) the survey aims to answer. The target population, measured variables and types of associations being investigated should be specific and unambiguous. Investigators should concentrate on what they ‘need to know’ rather than what would be ‘nice to know’. During development surveys should be piloted to identify problems. The main goal when implementing a survey is to maximize the response rate to avoid misleading results. Evidence-based strategies, including brief personalized surveys with stamped return envelopes, can be used to maximize the response rate. A poorly conducted survey can lead to misleading or invalid conclusions and may undermine participation in subsequent surveys by the target population.

Poor uptake of an online intervention in a cluster randomised controlled trial of online diabetes education for rural general practitioners

Background

In Australia, rural and remote communities have high rates of diabetes-related death and hospitalisation. General practitioners (GPs) play a major role in diabetes detection and management. Education of GPs could optimise diabetes management and improve patient outcomes at a population level. The study aimed to describe the uptake of a continuing medical education intervention for rural GPs and its impact on the viability of a cluster randomised controlled trial of the effects of continuing medical education on whole-town diabetes monitoring and control.

Method

Trial design: the cluster randomised controlled trial involved towns as the unit of allocation and analysis with outcomes assessed by de-identified pathology data (not reported here). The intervention programme consisted of an online active learning module, direct electronic access to specialist advice and performance feedback. Multiple rounds of invitation were used to engage GPs with the online intervention content. Evidence-based strategies (e.g. pre-notification, rewards, incentives) were incorporated into the invitations to enrol in the programme. Recruitment to the programme was electronically monitored through the hosting software package during the study intervention period.

Results

Eleven matched pairs of towns were included in the study. There were 146 GPs in the 11 intervention towns, of whom 34 (23.3%) enrolled in the programme, and 8 (5.5%) completed the online learning module. No town had more than 10% of the resident GPs complete the learning module. There were no contacts made by GPs regarding requests for specialist advice. Consequently, the trial was discontinued.

Conclusion

There is an ongoing need to engage primary care physicians in improving diabetes monitoring and management in rural areas. Online training options, while notionally attractive and accessible, are not likely to have high levels of uptake, even when evidence-based recruitment strategies are implemented.

Trial registration

Australian New Zealand Clinical Trials Registry, identifier: ACTRN12611000553976. Retrospectively registered on 31 May 2011.

Diabetes in rural towns: effectiveness of continuing education and feedback for healthcare providers in altering diabetes outcomes at a population level: protocol for a cluster randomised controlled trial

Background

Type 2 diabetes is one of the fastest growing chronic diseases internationally. The health complications associated with type 2 diabetes can be prevented, delayed, or improved via early diagnosis and effective management. This research aims to examine the impact of a primarily web-based educational intervention on the diabetes care provided by general practitioners (GPs) in rural areas, and subsequent patient outcomes. A population-level approach to outcome assessment is used, via whole-town de-identified pathology records.

Methods/design

The study uses a cluster randomised controlled trial with rural communities as the unit of analysis. Towns from four Australian states were selected and matched on factors including rurality, population size, proportion of the population who were Indigenous Australians, and socio-economic status. Eleven pairs of towns from two states were suitable for the trial, and one town from each pair was randomised to the experimental group. GPs in the towns allocated to the experimental group are offered an intervention package comprising education on best practice diabetes care via an on-line active learning module, a moderated discussion forum, access to targeted and specialist advice through an on-line request form, and town-based performance feedback on diabetes monitoring and outcomes. The package is offered via repeated direct mail.

Discussion

The benefits of the outcomes of the trial are described along with the challenges and limitations associated with the methodology.

Trial registration

Australian New Zealand Clinical Trials Registry: ACTRN12611000553976

Increasing response rates from physicians in oncology research: a structured literature review and data from a recent physician survey

Although the physician survey has become an important tool for oncology-focused health services research, such surveys often achieve low response rates. This mini-review reports the results of a structured review of the literature relating to increasing response rates for physician surveys, as well as our own experience from a survey of physicians as to their referral practices for suspected haematologic malignancy in the United States. PubMed and PsychINFO databases were used to identify methodological articles assessing factors that influence response rates for physician surveys; the results were tabulated and reviewed for trends. We also analysed the impact of a follow-up telephone call by a physician investigator to initial non-responders in our own mailed physician survey, comparing the characteristics of those who responded before vs after the call. The systematic review suggested that monetary incentives and paper (vs web or email) surveys increase response rates. In our own survey, follow-up telephone calls increased the response rate from 43.7% to 70.5%, with little discernible difference in the characteristics of early vs later responders. We conclude that in addition to monetary incentives and paper surveys, physician-to-physician follow-up telephone calls are an effective method to increase response rates in oncology-focused physician surveys.

Does an offer for a free on-line continuing medical education (CME) activity increase physician survey response rate? A randomized trial

BACKGROUND

Achieving a high response rate in a physician survey is challenging. Monetary incentives increase response rates but obviously add cost to a survey project. We wondered whether an offer of a free continuing medical education (CME) activity would be effective in improving survey response rate.

RESULTS

As part of a survey of a national sample of physicians, we randomized half to an offer for a free on-line CME activity upon completion of a web-based survey and the other half to no such offer. We compared response rates between the groups. A total of 1214 out of 8477 potentially eligible physicians responded to our survey, for an overall response rate of 14.3%. The response rate among the control group (no offer of CME credit) was 16.6%, while among those offered the CME opportunity, the response rate was 12.0% (p < 0.0001).

CONCLUSIONS

An offer for a free on-line CME activity did not improve physician survey response rate. On the contrary, the offer for a free CME activity actually appeared to worsen the response rate.

National survey of cardiologists' standard of practice for continuous ST-segment monitoring

BACKGROUND

Continuous ST-segment monitoring can be used to detect early and transient cardiac ischemia. The American Heart Association and American Association of Critical-Care Nurses recommend its use among specific patients, but such monitoring is routine practice in only about half of US hospitals.

OBJECTIVE

To determine cardiologists' awareness and practice standards regarding continuous ST-segment monitoring and the physicians' perceptions of appropriate patient selection, benefits and barriers, and usefulness of this technology.

METHODS

An electronic survey was sent to a random sample of 915 US cardiologists from a pool of 4985 certified cardiologists.

RESULTS

Of 200 responding cardiologists, 55% were unaware of the consensus guidelines. Of hospitals where respondents admitted patients, 49% had a standard of practice for using continuous ST-segment monitoring for cardiac patients. Most cardiologists agreed or strongly agreed that patients in the cardiovascular laboratory (87.5%) and intensive care unit (80.5%) should have such monitoring. Cardiologists routinely ordered ST monitoring for patients with acute coronary syndrome (67%) and after percutaneous coronary intervention (60%). The primary factor associated with higher perceptions for benefits, clinical usefulness, and past use of continuous ST-segment monitoring was whether or not hospitals in which cardiologists practiced had a standard of practice for using this monitoring. A secondary factor was awareness of published consensus guidelines for such monitoring.

CONCLUSION

Respondents (55%) were unaware of published monitoring guidelines. Hospital leaders could raise awareness by multidisciplinary review of evidence and possibly incorporating continuous ST-segment monitoring into hospitals' standards of practice.

Methods to increase response to postal and electronic questionnaires

BACKGROUND

Postal and electronic questionnaires are widely used for data collection in epidemiological studies but non-response reduces the effective sample size and can introduce bias. Finding ways to increase response to postal and electronic questionnaires would improve the quality of health research.

OBJECTIVES

To identify effective strategies to increase response to postal and electronic questionnaires.

SEARCH STRATEGY

We searched 14 electronic databases to February 2008 and manually searched the reference lists of relevant trials and reviews, and all issues of two journals. We contacted the authors of all trials or reviews to ask about unpublished trials. Where necessary, we also contacted authors to confirm methods of allocation used and to clarify results presented. We assessed the eligibility of each trial using pre-defined criteria.

SELECTION CRITERIA

Randomised controlled trials of methods to increase response to postal or electronic questionnaires.

DATA COLLECTION AND ANALYSIS

We extracted data on the trial participants, the intervention, the number randomised to intervention and comparison groups and allocation concealment. For each strategy, we estimated pooled odds ratios (OR) and 95% confidence intervals (CI) in a random-effects model. We assessed evidence for selection bias using Egger's weighted regression method and Begg's rank correlation test and funnel plot. We assessed heterogeneity among trial odds ratios using a Chi(2) test and the degree of inconsistency between trial results was quantified using the I(2) statistic.

MAIN RESULTS

PostalWe found 481 eligible trials. The trials evaluated 110 different ways of increasing response to postal questionnaires. We found substantial heterogeneity among trial results in half of the strategies. The odds of response were at least doubled using monetary incentives (odds ratio 1.87; 95% CI 1.73 to 2.04; heterogeneity P < 0.00001, I(2) = 84%), recorded delivery (1.76; 95% CI 1.43 to 2.18; P = 0.0001, I(2) = 71%), a teaser on the envelope - e.g. a comment suggesting to participants that they may benefit if they open it (3.08; 95% CI 1.27 to 7.44) and a more interesting questionnaire topic (2.00; 95% CI 1.32 to 3.04; P = 0.06, I(2) = 80%). The odds of response were substantially higher with pre-notification (1.45; 95% CI 1.29 to 1.63; P < 0.00001, I(2) = 89%), follow-up contact (1.35; 95% CI 1.18 to 1.55; P < 0.00001, I(2) = 76%), unconditional incentives (1.61; 1.36 to 1.89; P < 0.00001, I(2) = 88%), shorter questionnaires (1.64; 95% CI 1.43 to 1.87; P < 0.00001, I(2) = 91%), providing a second copy of the questionnaire at follow up (1.46; 95% CI 1.13 to 1.90; P < 0.00001, I(2) = 82%), mentioning an obligation to respond (1.61; 95% CI 1.16 to 2.22; P = 0.98, I(2) = 0%) and university sponsorship (1.32; 95% CI 1.13 to 1.54; P < 0.00001, I(2) = 83%). The odds of response were also increased with non-monetary incentives (1.15; 95% CI 1.08 to 1.22; P < 0.00001, I(2) = 79%), personalised questionnaires (1.14; 95% CI 1.07 to 1.22; P < 0.00001, I(2) = 63%), use of hand-written addresses (1.25; 95% CI 1.08 to 1.45; P = 0.32, I(2) = 14%), use of stamped return envelopes as opposed to franked return envelopes (1.24; 95% CI 1.14 to 1.35; P < 0.00001, I(2) = 69%), an assurance of confidentiality (1.33; 95% CI 1.24 to 1.42) and first class outward mailing (1.11; 95% CI 1.02 to 1.21; P = 0.78, I(2) = 0%). The odds of response were reduced when the questionnaire included questions of a sensitive nature (0.94; 95% CI 0.88 to 1.00; P = 0.51, I(2) = 0%).ElectronicWe found 32 eligible trials. The trials evaluated 27 different ways of increasing response to electronic questionnaires. We found substantial heterogeneity among trial results in half of the strategies. The odds of response were increased by more than a half using non-monetary incentives (1.72; 95% CI 1.09 to 2.72; heterogeneity P < 0.00001, I(2) = 95%), shorter e-questionnaires (1.73; 1.40 to 2.13; P = 0.08, I(2) = 68%), including a statement that others had responded (1.52; 95% CI 1.36 to 1.70), and a more interesting topic (1.85; 95% CI 1.52 to 2.26). The odds of response increased by a third using a lottery with immediate notification of results (1.37; 95% CI 1.13 to 1.65), an offer of survey results (1.36; 95% CI 1.15 to 1.61), and using a white background (1.31; 95% CI 1.10 to 1.56). The odds of response were also increased with personalised e-questionnaires (1.24; 95% CI 1.17 to 1.32; P = 0.07, I(2) = 41%), using a simple header (1.23; 95% CI 1.03 to 1.48), using textual representation of response categories (1.19; 95% CI 1.05 to 1.36), and giving a deadline (1.18; 95% CI 1.03 to 1.34). The odds of response tripled when a picture was included in an e-mail (3.05; 95% CI 1.84 to 5.06; P = 0.27, I(2) = 19%). The odds of response were reduced when "Survey" was mentioned in the e-mail subject line (0.81; 95% CI 0.67 to 0.97; P = 0.33, I(2) = 0%), and when the e-mail included a male signature (0.55; 95% CI 0.38 to 0.80; P = 0.96, I(2) = 0%).

AUTHORS' CONCLUSIONS

Health researchers using postal and electronic questionnaires can increase response using the strategies shown to be effective in this systematic review.

Methodologies for improving response rates in surveys of physicians: a systematic review

Although physician surveys are an important tool in health services and policy research, they are often characterized by low response rates. The authors conducted a systematic review of 66 published reports of efforts to improve response rates to physician surveys. Two general strategies were explored in this literature: incentive and design-based approaches. Even small financial incentives were found to be effective in improving physician response. Token nonmonetary incentives were much less effective. In terms of design strategies, postal and telephone strategies have generally been more successful than have fax or Web-based approaches, with evidence also supporting use of mixed-mode surveys in this population. In addition, use of first-class stamps on return envelopes and questionnaires designed to be brief, personalized, and endorsed by legitimizing professional associations were also more likely to be successful. Researchers should continue to implement design strategies that have been documented to improve the survey response of physicians.