Timing of electronic reminders did not improve trial participant questionnaire response: a randomized trial and meta-analyses

Undertaking Studies Within A Trial to evaluate recruitment and retention strategies for randomised controlled trials: lessons learnt from the PROMETHEUS research programme

Background

Randomised controlled trials ('trials') are susceptible to poor participant recruitment and retention. Studies Within A Trial are the strongest methods for testing the effectiveness of strategies to improve recruitment and retention. However, relatively few of these have been conducted.

Objectives

PROMoting THE Use of Studies Within A Trial aimed to facilitate at least 25 Studies Within A Trial evaluating recruitment or retention strategies. We share our experience of delivering the PROMoting THE Use of Studies Within A Trial programme, and the lessons learnt for undertaking randomised Studies Within A Trial.

Design

A network of 10 Clinical Trials Units and 1 primary care research centre committed to conducting randomised controlled Studies Within A Trial of recruitment and/or retention strategies was established. Promising recruitment and retention strategies were identified from various sources including Cochrane systematic reviews, the Study Within A Trial Repository, and existing prioritisation exercises, which were reviewed by patient and public members to create an initial priority list of seven recruitment and eight retention interventions. Host trial teams could apply for funding and receive support from the PROMoting THE Use of Studies Within A Trial team to undertake Studies Within A Trial. We also tested the feasibility of undertaking co-ordinated Studies Within A Trial, across multiple host trials simultaneously.

Setting

Clinical trials unit-based trials recruiting or following up participants in any setting in the United Kingdom were eligible.

Participants

Clinical trials unit-based teams undertaking trials in any clinical context in the United Kingdom.

Interventions

Funding of up to £5000 and support from the PROMoting THE Use of Studies Within A Trial team to design, implement and report Studies Within A Trial.

Main outcome measures

Number of host trials funded.

Results

Forty-two Studies Within A Trial were funded (31 host trials), across 12 Clinical Trials Units. The mean cost of a Study Within A Trial was £3535. Twelve Studies Within A Trial tested the same strategy across multiple host trials using a co-ordinated Study Within A Trial design, and four used a factorial design. Two recruitment and five retention strategies were evaluated in more than one host trial. PROMoting THE Use of Studies Within A Trial will add 18% more Studies Within A Trial to the Cochrane systematic review of recruitment strategies, and 79% more Studies Within A Trial to the Cochrane review of retention strategies. For retention, we found that pre-notifying participants by card, letter or e-mail before sending questionnaires was effective, as was the use of pens, and sending personalised text messages to improve questionnaire response. We highlight key lessons learnt to guide others planning Studies Within A Trial, including involving patient and public involvement partners; prioritising and selecting strategies to evaluate and elements to consider when designing a Study Within A Trial; obtaining governance approvals; implementing Studies Within A Trial, including individual and co-ordinated Studies Within A Trials; and reporting Study Within A Trials.

Limitations

The COVID-19 pandemic negatively impacted five Studies Within A Trial, being either delayed (n = 2) or prematurely terminated (n = 3).

Conclusions

PROMoting THE Use of Studies Within A Trial significantly increased the evidence base for recruitment and retention strategies. When provided with both funding and practical support, host trial teams successfully implemented Studies Within A Trial.

Future work

Future research should identify and target gaps in the evidence base, including widening Study Within A Trial uptake, undertaking more complex Studies Within A Trial and translating Study Within A Trial evidence into practice.

Study registration

All Studies Within A Trial in the PROMoting THE Use of Studies Within A Trial programme had to be registered with the Northern Ireland Network for Trials Methodology Research Study Within A Trial Repository.

Funding

This award was funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme (NIHR award ref: 13/55/80) and is published in full in Health Technology Assessment; Vol. 28, No. 2. See the NIHR Funding and Awards website for further award information.

Pre-notifications increase retention in a 17-year follow-up of adolescents born very preterm

OBJECTIVE

Retention is essential in follow-up studies to reduce missing data, which can cause bias and limit the generalizability of the results. We investigated whether pre-notification letters would increase the response rates of approval forms and questionnaires and reduce the need for post-notifications in a prospective follow-up study of 17-year-old adolescents.

STUDY DESIGN

and settings This long-term follow-up study included 269 adolescents were randomized (1:1) into a pre-notification group (n = 132) and a no pre-notification group (n = 137). The pre-notification letter was sent prior to the approval form and questionnaires. The outcome measures were the response rates to the approval forms and questionnaires and the rate of post-notifications required.

RESULTS

The adolescents who received the pre-notifications were more likely to return approval forms (n = 88/132, 67%) than the adolescents who did not receive the pre-notifications (n = 79/137, 58%) (OR 1.5, 95% CI 0.9-2.4). The rates of returned questionnaires were higher in the pre-notification group (n = 82/88, 93%) than in the no pre-notification group (n = 68/79, 86%) (OR 2.2, 95% CI 0.8-6.3). The adolescents who did not receive the pre-notifications were more likely to need the post-notifications than the adolescents who received the pre-notifications (OR 3.0, 95% CI 1.4 to 6.5).

CONCLUSIONS

Pre-notifications decreased the need for post-notifications and may increase retention in 17-year-old adolescents. Based on our findings, pre-notification letters are recommended in future follow-up studies in adolescents.

TRIAL REGISTRATION

The Ethics Review Committee of the Hospital District of South-West Finland approved the 17-year PIPARI Study protocol in January 2018 (23.1.2018; 2/180/2012). The study has been registered to the SWAT repository as SWAT 179. Filetoupload,1457904,en.pdf (qub.ac.uk).

Recruitment and retention interventions in surgical and wound care trials: A systematic review

BACKGROUND

Recruitment and retention to surgical trials has previously been reported to be problematic, resulting in research waste. Surgery often results in wounds, meaning these trials are likely to have similar populations. There is currently no systematic assessment of effective strategies for these populations and hence, systematic assessment of these was deemed to be of importance.

METHODS

A systematic review was conducted. Studies were eligible if they were randomised controlled trials undertaken to test an intervention to improve recruitment or retention within a surgical or wound based host randomised controlled trial. MEDLINE, EMBASE, Cochrane Library, ORRCA Database and the Northern Ireland Hub for Trials Methodology Research SWAT Repository Store were searched. Two independent reviewers screened the search results and extracted data for eligible studies using a piloted extraction form. A narrative synthesis was used due to a lack of heterogeneity between strategies which prevented meta-analysis.

RESULTS

A total of 2133 records were identified which resulted in 13 ultimately being included in the review; seven on recruitment and six on retention. All included studies were based within surgical host trials. Four of the seven recruitment studies focussed on the provision of consent information to participants, one focussed on study set up and one on staff training, with only one relating to consent information finding any significant effect. A range of retention strategies were assessed by the included studies, however only two found (pen vs no pen, mailing strategies) found any significant effect.

CONCLUSION

The included studies within a trial were all conducted within surgical trials. There was significant variation in strategies used, and limited replications and therefore further assessment may be warranted. Given the lack of studies embedded within wound care trials, further studies in this area are recommended.

TRIAL REGISTRATION

PROSPERO (CRD42020205475).

Pre-notification and personalisation of text messages to increase questionnaire completion in a smoking cessation pregnancy RCT: an embedded randomised factorial trial

BACKGROUND

Low completion rates of questionnaires in randomised controlled trials can compromise the reliability of the results, so ways to boost questionnaire completion are often implemented. Although there is evidence to suggest that sending a text message to participants increases completion, there is little evidence around the timing or personalisation of these text messages.  Methods:  A two-by-two factorial SWAT (study within a trial) was embedded within the MiQuit-3 trial, looking at smoking cessation within pregnant smokers. Participants who reached their 36-week gestational follow-up were randomised to receive a personalised or non-personalised text message, either one week or one day prior to their follow-up. Primary outcomes were completion rate of questionnaire via telephone. Secondary outcomes included: completion rate via any method, time to completion, and number of attempts to contact required.  Results  In total 194 participants were randomised into the SWAT to receive a text message that was personalised early(n=50), personalised late (n=47), non-personalised early(n=50), or non-personalised late(n=47). There was no evidence that timing of the text message (early: one week before; or late: one day before) had an effect on any of the outcomes. There was evidence that a personalised text message would result in fewer completions compared with a non-personalised text message when data was collected only via the telephone(adjusted OR 0.44, 95% CI 0.22-0.87, p=0.02). However, these results were not significant when looking at completion via any method (adjusted OR 0.61, 95% CI 0.30-1.24, p=0.17). There was no evidence to show that personalisation or not was better for any of the secondary outcomes.  Conclusion  Timing of the text message does not appear to influence the completion of questionnaires. Personalisation of a text message may be detrimental to questionnaire completion, if data is only collected via the telephone - however, more SWATs should be undertaken in this field.

Pre-notification and personalisation of text-messages to retain participants in a smoking cessation pregnancy RCT: an embedded randomised factorial trial

BACKGROUND

Low response rates in randomised controlled trials can compromise the reliability of the results, so ways to boost retention are often implemented. Although there is evidence to suggest that sending a text message to participants increases retention, there is little evidence around the timing or personalisation of these messages.  Methods:  A two-by-two factorial SWAT (study within a trial) was embedded within the MiQuit-3 trial, looking at smoking cessation within pregnant smokers. Participants who reached their 36-week gestational follow-up were randomised to receive a personalised or non-personalised text message, either one week or one day prior to the telephone follow-up. Primary outcomes were completion rate of questionnaire via telephone. Secondary outcomes included: completion rate via any method, time to completion, and number of reminders required.  Results  In total 194 participants were randomised into the SWAT; 50 to personalised early text, 47 to personalised late text, 50 to non-personalised early text, and 47 to non-personalised late text. There was no evidence that timing of the text message (early: one week before; or late: one day before) had an effect on any of the outcomes. There was evidence that a personalised text would result in fewer completions via telephone compared with a non-personalised text (adjusted OR 0.44, 95% CI 0.22-0.87, p=0.02). However, there was no evidence to show that personalisation or not was better for any of the secondary outcomes.  Conclusion  Timing of the text message does not appear to influence the retention of participants. Personalisation of a text message may be detrimental to retention; however, more SWATs should be undertaken in this field.

Study within a trial (SWAT) protocol. Investigating the effect of personalised versus non-personalised study invitations on recruitment: An embedded randomised controlled recruitment trial

Introduction

Recruitment into clinical trials is a common challenge experienced by healthcare researchers. Currently, there is little evidence regarding strategies to improve recruitment into clinical trials. However, preliminary research suggests the personalisation of study invitation letters may increase recruitment rates. As such, there is a need to investigate the effectiveness of personalisation strategies on trial recruitment rates. This study within a trial (SWAT) will investigate the effect of personalised versus non-personalised study invitation letters on recruitment rates into the host trial ENGAGE, a feasibility study of an internet-administered, guided, Cognitive-Behavioural Therapy (CBT) based self-help intervention for parents of children previously treated for cancer.

Methods

An embedded randomised controlled trial (RCT) will investigate the effectiveness of a personalised study invitation letter including the potential participant’s name and address compared with a standard, non-personalised letter without name or address, on participant recruitment rates into the ENGAGE study. The primary outcome is differences in the proportion of participants recruited, examined using logistic regression. Results will be reported as adjusted odds ratios with 95% confidence intervals.

Discussion

Even moderate effects of the personalisation of study invitation letters on recruitment rates could be of significant value by shortening study length, saving resources, and providing a faster answer to the clinical question posed by the study. This protocol can be used as a template for other researchers who wish to contribute to the evidence base for trial decision-making, by embedding a similar SWAT into their trial.

Trial registration

ISRCTN 57233429; ISRCTN 18404129; SWAT 112, Northern Ireland Hub for Trials Methodology Research SWAT repository (2018 OCT 1 1231).