Including a pen and/or cover letter, containing social incentive text, had no effect on questionnaire response rate: a factorial randomised controlled Study within a Trial

Trial Forge Guidance 4: a guideline for reporting the results of randomised Studies Within A Trial (SWATs)

BACKGROUND

Evidence to support decisions on trial processes is minimal. One way to generate this evidence is to use a Study Within A Trial (SWAT) to test trial processes or explore methodological uncertainties. SWAT evidence relies on replication to ensure sufficient power and broad applicability of findings. Prompt reporting is therefore essential; however, SWAT publications are often the first to be abandoned in the face of other time pressures. Reporting guidance for embedded methodology trials does exist but is not widely used. We sought therefore to build on these guidelines to develop a straightforward, concise reporting standard, which remains adherent to the CONSORT guideline.

METHODS

An iterative process was used to develop the guideline. This included initial meetings with key stakeholders, development of an initial guideline, pilot testing of draft guidelines, further iteration and pilot testing, and finalisation of the guideline.

RESULTS

We developed a reporting guideline applicable to randomised SWATs, including replications of previous evaluations. The guideline follows the Consolidated Standards for Reporting Trials (CONSORT) statement and provides example text to ensure ease and clarity of reporting across all domains.

CONCLUSIONS

The SWAT reporting guideline will aid authors, reviewers, and journal editors to produce and review clear, structured reports of randomised SWATs, whilst also adhering to the CONSORT guideline.

TRIAL REGISTRATION

EQUATOR Network - Guidelines Under Development ( https://www.equator-network.org/library/reporting-guidelines-under-development/reporting-guidelines-under-development-for-clinical-trials/#SWAT ). Registered on 25 March 2021.

Undertaking Studies Within A Trial to evaluate recruitment and retention strategies for randomised controlled trials: lessons learnt from the PROMETHEUS research programme

Background

Randomised controlled trials ('trials') are susceptible to poor participant recruitment and retention. Studies Within A Trial are the strongest methods for testing the effectiveness of strategies to improve recruitment and retention. However, relatively few of these have been conducted.

Objectives

PROMoting THE Use of Studies Within A Trial aimed to facilitate at least 25 Studies Within A Trial evaluating recruitment or retention strategies. We share our experience of delivering the PROMoting THE Use of Studies Within A Trial programme, and the lessons learnt for undertaking randomised Studies Within A Trial.

Design

A network of 10 Clinical Trials Units and 1 primary care research centre committed to conducting randomised controlled Studies Within A Trial of recruitment and/or retention strategies was established. Promising recruitment and retention strategies were identified from various sources including Cochrane systematic reviews, the Study Within A Trial Repository, and existing prioritisation exercises, which were reviewed by patient and public members to create an initial priority list of seven recruitment and eight retention interventions. Host trial teams could apply for funding and receive support from the PROMoting THE Use of Studies Within A Trial team to undertake Studies Within A Trial. We also tested the feasibility of undertaking co-ordinated Studies Within A Trial, across multiple host trials simultaneously.

Setting

Clinical trials unit-based trials recruiting or following up participants in any setting in the United Kingdom were eligible.

Participants

Clinical trials unit-based teams undertaking trials in any clinical context in the United Kingdom.

Interventions

Funding of up to £5000 and support from the PROMoting THE Use of Studies Within A Trial team to design, implement and report Studies Within A Trial.

Main outcome measures

Number of host trials funded.

Results

Forty-two Studies Within A Trial were funded (31 host trials), across 12 Clinical Trials Units. The mean cost of a Study Within A Trial was £3535. Twelve Studies Within A Trial tested the same strategy across multiple host trials using a co-ordinated Study Within A Trial design, and four used a factorial design. Two recruitment and five retention strategies were evaluated in more than one host trial. PROMoting THE Use of Studies Within A Trial will add 18% more Studies Within A Trial to the Cochrane systematic review of recruitment strategies, and 79% more Studies Within A Trial to the Cochrane review of retention strategies. For retention, we found that pre-notifying participants by card, letter or e-mail before sending questionnaires was effective, as was the use of pens, and sending personalised text messages to improve questionnaire response. We highlight key lessons learnt to guide others planning Studies Within A Trial, including involving patient and public involvement partners; prioritising and selecting strategies to evaluate and elements to consider when designing a Study Within A Trial; obtaining governance approvals; implementing Studies Within A Trial, including individual and co-ordinated Studies Within A Trials; and reporting Study Within A Trials.

Limitations

The COVID-19 pandemic negatively impacted five Studies Within A Trial, being either delayed (n = 2) or prematurely terminated (n = 3).

Conclusions

PROMoting THE Use of Studies Within A Trial significantly increased the evidence base for recruitment and retention strategies. When provided with both funding and practical support, host trial teams successfully implemented Studies Within A Trial.

Future work

Future research should identify and target gaps in the evidence base, including widening Study Within A Trial uptake, undertaking more complex Studies Within A Trial and translating Study Within A Trial evidence into practice.

Study registration

All Studies Within A Trial in the PROMoting THE Use of Studies Within A Trial programme had to be registered with the Northern Ireland Network for Trials Methodology Research Study Within A Trial Repository.

Funding

This award was funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme (NIHR award ref: 13/55/80) and is published in full in Health Technology Assessment; Vol. 28, No. 2. See the NIHR Funding and Awards website for further award information.

Enclosing a pen in a postal questionnaire follow-up to increase response rate: a study within a trial

Background

Poor response rates to follow-up questionnaires can adversely affect the progress of a randomised controlled trial and the validity of its results. This embedded 'study within a trial' aimed to investigate the impact of including a pen with the postal 3-month questionnaire completed by the trial participants on the response rates to this questionnaire.

Methods

This study was a two-armed randomised controlled trial nested in the Gentle Years Yoga (GYY) trial. Participants in the intervention group of the GYY trial were allocated 1:1 using simple randomisation to either receive a pen (intervention) or no pen with their 3-month questionnaire (control). The primary outcome was the proportion of participants sent a 3-month questionnaire who returned it. Secondary outcomes were time taken to return the questionnaire, proportion of participants sent a reminder to return the questionnaire, and completeness of the questionnaire. Binary outcomes were analysed using logistic regression, time to return by Cox Proportional Hazards regression and number of items completed by linear regression.

Results

There were 111 participants randomised to the pen group and 118 to the no pen group who were sent a 3-month questionnaire. There was no evidence of a difference in return rates between the two groups (pen 107 (96.4%), no pen 117 (99.2%); OR 0.23, 95% CI 0.02 to 2.19, p=0.20). Furthermore, there was no evidence of a difference between the two groups in terms of time to return the questionnaire (HR 0.90, 95% CI 0.69 to 1.18, p=0.47), the proportion of participants sent a reminder (OR 0.85, 95% CI 0.48 to 1.53, p=0.60) nor the number of items completed (mean difference 0.51, 95% CI -0.04 to 1.06, p=0.07).

Conclusion

The inclusion of a pen with the postal 3-month follow-up questionnaire did not have a statistically significant effect on response rate.

Enclosing a pen in a postal questionnaire follow-up to increase response rate: a study within a trial [version 1; peer review: 2 approved]

Background

Poor response rates to follow-up questionnaires can adversely affect the progress of a randomised controlled trial and the validity of its results. This embedded 'study within a trial' aimed to investigate the impact of including a pen with the postal 3-month questionnaire completed by the trial participants on the response rates to this questionnaire.

Methods

This study was a two-armed randomised controlled trial nested in the Gentle Years Yoga (GYY) trial. Participants in the intervention group of the GYY trial were allocated 1:1 using simple randomisation to either receive a pen (intervention) or no pen with their 3-month questionnaire (control). The primary outcome was the proportion of participants sent a 3-month questionnaire who returned it. Secondary outcomes were time taken to return the questionnaire, proportion of participants sent a reminder to return the questionnaire, and completeness of the questionnaire. Binary outcomes were analysed using logistic regression, time to return by Cox Proportional hazards regression and number of items completed by linear regression.

Results

There were 111 participants randomised to the pen group and 118 to the no pen group who were sent a 3-month questionnaire. There was no evidence of a difference in return rates between the two groups (pen 107 (96.4%), no pen 117 (99.2%); OR 0.23, 95% CI 0.02 to 2.19, p=0.20). Furthermore, there was no evidence of a difference between the two groups in terms of time to return the questionnaire (HR 0.90, 95% CI 0.69 to 1.18, p=0.47), the proportion of participants sent a reminder (OR 0.85, 95% CI 0.48 to 1.53, p=0.60) nor the number of items completed (mean difference 0.51, 95% CI-0.04 to 1.06, p=0.07).

Conclusion

The inclusion of a pen with the postal 3-month follow-up questionnaire did not have a statistically significant effect on response rate.