Making randomised trials more efficient: report of the first meeting to discuss the Trial Forge platform

Facilitators and barriers to the delivery of the PARAMEDIC2 trial

Background

PARAMEDIC2 was a medicines trial comparing adrenaline with placebo in out-of-hospital cardiac arrest (OHCA). At the time, United Kingdom (UK) Emergency Medical Systems (EMS) were inexperienced in delivering such research.

Aim

To identify barriers and facilitators to delivery of the PARAMEDIC2 (Adrenaline) trial by five UK NHS EMS.

Methods

This qualitative study took a grounded theory approach to thematic analysis of workshop data. Members of the trial teams from each service attended a workshop in November 2018 and discussed their experiences in answer to two prompt questions. Data were coded and themes presented.

Results

Three main themes were identified: professionalism, organisational investment and unique features of EMS. The study provided an opportunity for recruiting paramedics and research paramedics to demonstrate their professionalism. Research paramedics felt it was part of their professional duty to initiate discussions with the patient/family regarding the trial rather than leave this task to the hospital teams as would usually happen. Organisational investment was reflected by prioritising trial training and further development of research paramedics. By these means, research culture was developed. The unique features of EMS such as geographical challenges were often addressed with technological solutions and through building relationships with internal teams.

Conclusion

Barriers to trial delivery included infrequent exposure to the condition of interest and lack of continuity in research paramedic roles. Facilitators identified included flexibility of the research protocol, and organisational investment in the development of research paramedics.Participating in PARAMEDIC2 was challenging for the EMS involved, but ultimately strengthened their research culture.

Undertaking Studies Within A Trial to evaluate recruitment and retention strategies for randomised controlled trials: lessons learnt from the PROMETHEUS research programme

Background

Randomised controlled trials ('trials') are susceptible to poor participant recruitment and retention. Studies Within A Trial are the strongest methods for testing the effectiveness of strategies to improve recruitment and retention. However, relatively few of these have been conducted.

Objectives

PROMoting THE Use of Studies Within A Trial aimed to facilitate at least 25 Studies Within A Trial evaluating recruitment or retention strategies. We share our experience of delivering the PROMoting THE Use of Studies Within A Trial programme, and the lessons learnt for undertaking randomised Studies Within A Trial.

Design

A network of 10 Clinical Trials Units and 1 primary care research centre committed to conducting randomised controlled Studies Within A Trial of recruitment and/or retention strategies was established. Promising recruitment and retention strategies were identified from various sources including Cochrane systematic reviews, the Study Within A Trial Repository, and existing prioritisation exercises, which were reviewed by patient and public members to create an initial priority list of seven recruitment and eight retention interventions. Host trial teams could apply for funding and receive support from the PROMoting THE Use of Studies Within A Trial team to undertake Studies Within A Trial. We also tested the feasibility of undertaking co-ordinated Studies Within A Trial, across multiple host trials simultaneously.

Setting

Clinical trials unit-based trials recruiting or following up participants in any setting in the United Kingdom were eligible.

Participants

Clinical trials unit-based teams undertaking trials in any clinical context in the United Kingdom.

Interventions

Funding of up to £5000 and support from the PROMoting THE Use of Studies Within A Trial team to design, implement and report Studies Within A Trial.

Main outcome measures

Number of host trials funded.

Results

Forty-two Studies Within A Trial were funded (31 host trials), across 12 Clinical Trials Units. The mean cost of a Study Within A Trial was £3535. Twelve Studies Within A Trial tested the same strategy across multiple host trials using a co-ordinated Study Within A Trial design, and four used a factorial design. Two recruitment and five retention strategies were evaluated in more than one host trial. PROMoting THE Use of Studies Within A Trial will add 18% more Studies Within A Trial to the Cochrane systematic review of recruitment strategies, and 79% more Studies Within A Trial to the Cochrane review of retention strategies. For retention, we found that pre-notifying participants by card, letter or e-mail before sending questionnaires was effective, as was the use of pens, and sending personalised text messages to improve questionnaire response. We highlight key lessons learnt to guide others planning Studies Within A Trial, including involving patient and public involvement partners; prioritising and selecting strategies to evaluate and elements to consider when designing a Study Within A Trial; obtaining governance approvals; implementing Studies Within A Trial, including individual and co-ordinated Studies Within A Trials; and reporting Study Within A Trials.

Limitations

The COVID-19 pandemic negatively impacted five Studies Within A Trial, being either delayed (n = 2) or prematurely terminated (n = 3).

Conclusions

PROMoting THE Use of Studies Within A Trial significantly increased the evidence base for recruitment and retention strategies. When provided with both funding and practical support, host trial teams successfully implemented Studies Within A Trial.

Future work

Future research should identify and target gaps in the evidence base, including widening Study Within A Trial uptake, undertaking more complex Studies Within A Trial and translating Study Within A Trial evidence into practice.

Study registration

All Studies Within A Trial in the PROMoting THE Use of Studies Within A Trial programme had to be registered with the Northern Ireland Network for Trials Methodology Research Study Within A Trial Repository.

Funding

This award was funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme (NIHR award ref: 13/55/80) and is published in full in Health Technology Assessment; Vol. 28, No. 2. See the NIHR Funding and Awards website for further award information.

Making implementation science more efficient: capitalizing on opportunities beyond the field

Implementation researchers often find themselves as research partners in practice improvement projects, clinical trials or other applied health studies. The implementation science component in these projects can be described as supportive, descriptive or explanatory. This commentary reflects on the potential contributions of such projects to implementation science. They may provide evidence on implementation strategies, so it is essential to identify and evaluate these separately from the clinical and preventive interventions of interest. The use of theory on implementation processes and associated factors can contribute to knowledge accumulation, particularly if the focus is on what actually gets implemented when, why and how. The development and validation of relevant measures is a third potential contribution to implementation science. Although not all issues in implementation science can be addressed in this way, capitalization on the opportunities beyond the field can contribute to implementation science.

Endovascular Management of Brain Arteriovenous Malformations

Due to the risk of cerebral hemorrhage, and its related morbidity-mortality, brain arteriovenous malformations (bAVMs) are a rare and potentially life-threatening disease. Despite this, there is only one randomized controlled trial on bAVM management, A Randomized trial of Unruptured Brain Arteriovenous malformations (ARUBA). The results of the ARUBA trial favor a noninterventional approach in the case of an unruptured bAVM; however, implementation of these findings is challenging in daily practice. Instead, management of bAVM relies on multidisciplinary discussions that lead to patient-specific strategies based on patient preferences, local expertise, and experience in referral centers. Considering the diverse patterns of presentation and numerous treatment modalities, implementing standardized guidelines in this context proves challenging, notwithstanding the recommendations or expert opinions offered. Endovascular treatment (EVT) of bAVM can be curative, or can serve as an adjunct treatment prior to surgery or radiosurgery ("pre-EVT"). EVT practice is in constant evolution (i.e., venous approach, combination with surgery during the same anesthesia, etc.). Liquid embolic agents such as ethylene vinyl alcohol (EVOH) copolymer and cyanoacrylates (CYA), and their method of injection to increase bAVM occlusion have also benefited from technical evolutions such as the use of adjunctive flow arrest techniques (mini balloons, pressure cooker technique, and multiple catheters). Further research is necessary to evaluate the advantages and disadvantages of EVT for bAVM.

The high-volume haemodiafiltration vs high-flux haemodialysis registry trial (H4RT): a multi-centre, unblinded, randomised, parallel-group, superiority study to compare the effectiveness and cost-effectiveness of high-volume haemodiafiltration and high-flux haemodialysis in people with kidney failure on maintenance dialysis using linkage to routine healthcare databases for outcomes

BACKGROUND

More than a third of the 65,000 people living with kidney failure in the UK attend a dialysis unit 2-5 times a week to have their blood cleaned for 3-5 h. In haemodialysis (HD), toxins are removed by diffusion, which can be enhanced using a high-flux dialyser. This can be augmented with convection, as occurs in haemodiafiltration (HDF), and improved outcomes have been reported in people who are able to achieve high volumes of convection. This study compares the clinical- and cost-effectiveness of high-volume HDF compared with high-flux HD in the treatment of kidney failure.

METHODS

This is a UK-based, multi-centre, non-blinded randomised controlled trial. Adult patients already receiving HD or HDF will be randomised 1:1 to high-volume HDF (aiming for 21+ L of substitution fluid adjusted for body surface area) or high-flux HD. Exclusion criteria include lack of capacity to consent, life expectancy less than 3 months, on HD/HDF for less than 4 weeks, planned living kidney donor transplant or home dialysis scheduled within 3 months, prior intolerance of HDF and not suitable for high-volume HDF for other clinical reasons. The primary outcome is a composite of non-cancer mortality or hospital admission with a cardiovascular event or infection during follow-up (minimum 32 months, maximum 91 months) determined from routine data. Secondary outcomes include all-cause mortality, cardiovascular- and infection-related morbidity and mortality, health-related quality of life, cost-effectiveness and environmental impact. Baseline data will be collected by research personnel on-site. Follow-up data will be collected by linkage to routine healthcare databases - Hospital Episode Statistics, Civil Registration, Public Health England and the UK Renal Registry (UKRR) in England, and equivalent databases in Scotland and Wales, as necessary - and centrally administered patient-completed questionnaires. In addition, research personnel on-site will monitor for adverse events and collect data on adherence to the protocol (monthly during recruitment and quarterly during follow-up).

DISCUSSION

This study will provide evidence of the effectiveness and cost-effectiveness of HD as compared to HDF for adults with kidney failure in-centre HD or HDF. It will inform management for this patient group in the UK and internationally.

TRIAL REGISTRATION

ISRCTN10997319 . Registered on 10 October 2017.

The effect of personalised versus non-personalised study invitations on recruitment within the ENGAGE feasibility trial: an embedded randomised controlled recruitment trial

BACKGROUND

Recruitment into clinical trials is challenging and there is a lack of evidence on effective recruitment strategies. Personalisation of invitation letters is a potentially pragmatic and feasible way of increasing recruitment rates at a low-cost. However, there is a lack of evidence concerning the effect of personalising of study invitation letters on recruitment rates.

METHODS

We undertook a Study Within A Trial (SWAT) to investigate the effect of personalised versus non-personalised study invitation letters on recruitment rates into the host feasibility trial ENGAGE, a feasibility study of an internet-administered, guided, Low Intensity Cognitive-Behavioural Therapy based self-help intervention for parents of children previously treated for cancer. An intervention group (n = 254) received a personalised study invitation letter and the control group (n = 255) received a non-personalised study invitation letter. The primary outcome was the proportion of participants in the intervention group and the control group enrolled into the ENGAGE host feasibility trial. Secondary outcomes relating to the recruitment and screening process, and retention were examined. Differences in proportions between groups for the primary and secondary outcomes were estimated using logistic regression.

RESULTS

Of the 509 potential participants, 56 (11.0%) were enrolled into the ENGAGE host feasibility trial: personalised: 30/254 (11.8%) and non-personalised: 26/255 (10.2%). No statistically significant effect on personalisation of enrolment was found (OR 1.18, 95% CI 0.68-2.06). No statistically significant differences were found for any secondary outcome.

CONCLUSIONS

Personalisation of study invitations had no effect on recruitment. However, given the small study sample size in the present SWAT, and lack of similar embedded recruitment RCTs to enable a meta-analysis, additional SWATs to examine the personalisation of study invitation letters are warranted.

TRIAL REGISTRATION

ISRCTN57233429 ; ISRCTN18404129 ; SWAT 112, Northern Ireland Hub for Trials Methodology Research SWAT repository (2018 OCT 1 1231) ( https://www.qub.ac.uk/sites/TheNorthernIrelandNetworkforTrialsMethodologyResearch/FileStore/Filetoupload,939618,en.pdf ).

Considerations for clinical trial design and conduct in the evaluation of novel advanced therapeutics in neurodegenerative disease

The recent advances in the development of potentially disease modifying cell and gene therapies for neurodegenerative disease has resulted in the production of a number of promising novel therapies which are now moving forward to clinical evaluation. The robust evaluation of these therapies pose a significant number of challenges when compared to more traditional evaluations of pharmacotherapy, which is the current mainstay of neurodegenerative disease symptom management. Indeed, there is an inherent complexity in the design and conduct of these trials at multiple levels. Here we discuss specific aspects requiring consideration in the context of investigating novel cell and gene therapies for neurodegenerative disease. This extends to overarching trial designs that could be employed and the factors that underpin design choices such outcome assessments, participant selection and methods for delivery of cell and gene therapies. We explore methods of data collection that may improve efficiency in trials of cell and gene therapy to maximize data sharing and collaboration. Lastly, we explore some of the additional context beyond efficacy evaluations that should be considered to ensure implementation across relevant healthcare settings.

Conducting Digital Health Care Research: Document Analysis of Challenges Experienced During Intervention Development and Feasibility Study Setup of an Internet-Administered Intervention for Parents of Children Treated for Cancer

Background

The design and conduct of research to develop, test, and evaluate complex health care interventions is challenging. Although the existing literature describes key challenges associated with the design and conduct of definitive (evaluation) trials, there is a lack of information concerning specific challenges associated with the intervention development phase and setup of feasibility studies. In particular, the literature is scarce concerning the challenges associated with conducting digital health care research, such as research on internet-administered interventions and research using digital features to support the execution of study procedures (eg, recruitment, consent, retention, and data collection and management). This study is conducted in the context of the intervention development and feasibility study setup phases of an internet-administered, guided, low-intensity cognitive behavioral therapy–based intervention for parents of children previously treated for cancer.

Objective

The aim of this study is to explore the challenges experienced during the development phase of the internet-administered intervention and digital features to support the execution of the study procedures and a feasibility study setup.

Methods

To explore the key challenges experienced, we conducted a document analysis of written records from all study meetings held by the research team (meeting minutes) between June 7, 2018, and January 10, 2020, guided by a thematic analysis approach. Furthermore, discussion groups with members of the research team were held to develop a more detailed understanding of the key challenges experienced. Methods and results are reported in accordance with the relevant items from the Standards for Reporting Qualitative Research checklist.

Results

Six main themes were identified: decision-making and communication, expertise, external constraints, flexibility, planning and scheduling, and technical constraints.

Conclusions

Significant challenges were experienced during the intervention development and setup phases of the feasibility study. Implications are discussed to inform future design, conduct, and planning of internet-administered intervention development and feasibility studies, especially within the context of digital health care research.

Trainee Principal Investigator Could Improve Recruitment in Trauma Trials: Review of Literature and Experience From a Trauma Center

INTRODUCTION

Recruitment of patients to participate in randomized control trials (RCT) is a challenging task, especially for trauma trials in which the identification and recruitment are time-limited. Multiple strategies have been tried to improve the participation of doctors and the recruitment of patients. The aim was to study the effect of a trainee principal investigator (TPI) on the efficacy of recruitment for a multicenter hip fracture RCT.

METHODS

A retrospective study comparing the number of junior doctors participating in the WHiTE 8 COPAL RCT and patients recruited before and after the introduction of formal TPI role at a major trauma center in the UK. Data was collected for nine months "before" (Nov 2018-July 2019) and six months "after" (Sept 2019-Feb 2020) the role of TPI was assigned.

RESULTS

From November 2018 to February 2020, a total of 292 patients were eligible for recruitment into this trial, out of which 196 (67.12 %) were successfully recruited. Excluding the research team, there were seven junior doctors actively recruiting in the "before period" in comparison with 10 in the "after period." Significantly more patients were recruited by junior doctors after a TPI was assigned. Overall, more percentage of eligible patients were recruited into the trial after a TPI was assigned, and this was statistically significant.

CONCLUSION

The allocation of a formal TPI significantly improved the recruitment of patients in a national RCT. TPI can work alongside the principal investigator and research team to be a valuable link person coordinating and engaging local trainees to take part in trials.

Rationale and protocol for a cluster randomized, cross-over trial of recruitment methods of rural children in primary care clinics: A feasibility study of a pediatric weight control trial in the IDeA States Pediatric Clinical Trials Network

A significant percentage of clinical trials fail due to poor recruitment. Despite this, few studies exist to evaluate clinical trial recruitment strategies using a randomized approach in any population, and none exist to test recruitment strategies for trials involving children or rural populations. For clinical trials focused on weight control, evaluating retention and dose are extremely important, as poor retention can lead to biased samples and existing research shows that dose (i.e. contact hours) is directly related to patient outcome. Finally, adequacy of blinding of assessment teams is rarely reported in pediatric trials, and unblinded staff may be more likely to inadvertently bias findings. Therefore, in this feasibility trial we aim to use rigorous clinical trial methodology to assess the effectiveness of two different recruitment strategies, as well as test retention, dose, and blinding. Specifically, we describe the rationale, design, and planned implementation of a feasibility study of a rural pediatric obesity treatment trial that will be implemented in four medical clinics in four states affiliated with the Environmental influences on Child Health Outcomes IDeA States Pediatric Clinical Trials Network (ECHO ISPCTN). The primary objective is to assess recruitment rate for consecutive recruitment (approaching recently seen eligible patients in consecutive order by date seen) compared to traditional recruitment (such as posters, flyers, tear-offs), as well as to assess retention, dose, and blinding. If successful, this trial will support the implementation of a large multi-state trial directed at addressing obesity in rural children and their families recruited from their primary care clinics. Registered with ClinicalTrials.gov NCT ID NCT04142034.

Test evaluation trials present different challenges for trial managers compared to intervention trials

INTRODUCTION

Test evaluation trials present different challenges for trial managers compared to intervention trials. There has been very little research on the management of test evaluation trials and how this impacts on trial success, in comparison with intervention trials. Evaluations of medical tests present specific challenges, because they are a pivot point bridging the complexities of pathways prompting testing with treatment decision-making. We systematically explored key differences in the trial design and management of test evaluation trials compared to intervention trials at the different stages of study design and delivery. We identified challenges in test evaluation trials that were more pronounced than in intervention trials, based on experience from 10 test evaluation trials.

METHODS

We formed a focus group of 7 trial managers and a statistician who had been involved in the day-to-day management of both test evaluation trials and intervention trials. We used discussion and content analysis to group challenges from 10 trials into a structured thematic format. The trials covered a range of medical conditions, diagnostic tests, clinical pathways and conditions including chronic kidney disease, chronic pelvic pain, colitis, detrusor over-activity, group B streptococcal colonisation, tuberculosis and colorectal, lung, ovarian and thyroid cancers.

RESULTS

We identified 10 common themes underlying challenges that are more pronounced in test evaluation compared to intervention trials. We illustrate these themes with examples from 10 trials, including with 31 specific challenges we experienced. The themes were ethics/governance; accessing patient populations; recruitment; patient preference; test processes, clinical pathways and samples storage; uncertainty of diagnostic results; verifying diagnosis (reference standard); follow-up; adverse effects; and diagnostic impact.

CONCLUSION

We present 10 common themes, including 31 challenges, in test evaluation trials that will be helpful to others designing and managing future test evaluation trials. Proactive identification of potential challenges at the design and planning stages of test evaluation trials will enable strategies to improve trial design and management that may be different from standard strategies used for intervention trials. Future work could extend this topic to include challenges for other trial stakeholders including participants, clinicians, statisticians and funders.

TRIAL REGISTRATION

All trials reviewed in this project were registered and are provided in Table 1.

Redundant trials can be prevented, if the EU clinical trial regulation is applied duly

The problem of wasteful clinical trials has been debated relentlessly in the medical community. To a significant extent, it is attributed to redundant trials - studies that are carried out to address questions, which can be answered satisfactorily on the basis of existing knowledge and accessible evidence from prior research. This article presents the first evaluation of the potential of the EU Clinical Trials Regulation 536/2014, which entered into force in 2014 but is expected to become applicable at the end of 2021, to prevent such trials. Having reviewed provisions related to the trial authorisation, we propose how certain regulatory requirements for the assessment of trial applications can and should be interpreted and applied by national research ethics committees and other relevant authorities in order to avoid redundant trials and, most importantly, preclude the unnecessary recruitment of trial participants and their unjustified exposure to health risks.

Does a video clip enhance recruitment into a parenting trial? Learnings from a study within a trial

Background

Reaching recruitment targets in randomised controlled trials is a challenge. Media tools are increasingly used to engage participants, yet there is a paucity of research into the use of video to optimise recruitment. We therefore tested whether adding a participant information video clip to a standard participant information sheet improved recruitment into a parenting trial.

Methods

One hundred seven participants were randomised to receive either a participant information sheet (n = 51) or an informational video clip (n = 56) as part of an email contact following a screening phase. All participants went on to receive the information sheet as part of the existing consent procedure.

Results

The video condition did not increase the odds of recruitment into the trial, such that those in the video condition were significantly less likely to participate in the main trial (OR = 0.253, CI = 0.104–0.618, p = 0.003).

Conclusion

The introduction of a video clip into the recruitment stages of a parenting trial did not lead to an improvement in recruitment; however, the small sample size precludes definitive inferences. We offer reflections on challenges encountered in implementing the SWAT and suggestions for other researchers seeking to embed recruitment SWATs into similar trials.

Trial registration

Current controlled trials ISRCTN 58327365. Registered on 19 March 2015.

SWAT registration

SWAT 106; Effects of a video clip on recruitment into a randomised trial. Registered on 20 December 2016.

Stroke rehabilitation research translation in Australia: a survey of clinical trialists

PURPOSE

Translation of findings from stroke trials into clinical practice remains low. Little is known about planned translation activities from the perspective of trialists who generate the evidence. This study aims to investigate perceptions of Australian stroke clinical trialists' about implementation of their findings into practice, and what translation activities they embedded into trial protocols.

MATERIALS AND METHOD

A descriptive cohort design and electronic survey was conducted. Three databases were searched to identify Australian stroke rehabilitation trials published between 2007 and 2017. Corresponding authors of the included trials were invited to complete an anonymous online survey about implementation of their trial intervention.

RESULTS

Fifty-one trialists were invited to participate and 38 completed the survey (74% response rate). The majority (79%) considered their trial results to be clinically significant and 68% had pre-planned knowledge translation activities. The most common planned translation activities were publication (89%), conference presentation (87%), and feedback of results to target audiences (58%). Mixed opinions were evident regarding when and how to design knowledge translation activities for inclusion in trial protocols. Stroke rehabilitation trialists rely mostly on anecdotal reports about implementation of trial interventions, with few formally measuring uptake.Implications for rehabilitationThe most common knowledge translation activities planned for by researchers are publication of findings in academic journals, and conference presentations.Mixed opinions exist regarding when and how to design for knowledge translation activities in trial protocols.Limited formal measurement of intervention uptake is completed.Further implementation research is required to support and measure effective translation efforts when embedded into trial protocols.

Prediction of RECRUITment In randomized clinical Trials (RECRUIT-IT)-rationale and design for an international collaborative study

BACKGROUND

Poor recruitment of patients is the predominant reason for early termination of randomized clinical trials (RCTs). Systematic empirical investigations and validation studies of existing recruitment models, however, are lacking. We aim to provide evidence-based guidance on how to predict and monitor recruitment of patients into RCTs. Our specific objectives are the following: (1) to establish a large sample of RCTs (target n = 300) with individual patient recruitment data from a large variety of RCTs, (2) to investigate participant recruitment patterns and study site recruitment patterns and their association with the overall recruitment process, (3) to investigate the validity of a freely available recruitment model, and (4) to develop a user-friendly tool to assist trial investigators in the planning and monitoring of the recruitment process.

METHODS

Eligible RCTs need to have completed the recruitment process, used a parallel group design, and investigated any healthcare intervention where participants had the free choice to participate. To establish the planned sample of RCTs, we will use our contacts to national and international RCT networks, clinical trial units, and individual trial investigators. From included RCTs, we will collect patient-level information (date of randomization), site-level information (date of trial site activation), and trial-level information (target sample size). We will examine recruitment patterns using recruitment trajectories and stratifications by RCT characteristics. We will investigate associations of early recruitment patterns with overall recruitment by correlation and multivariable regression. To examine the validity of a freely available Bayesian prediction model, we will compare model predictions to collected empirical data of included RCTs. Finally, we will user-test any promising tool using qualitative methods for further tool improvement.

DISCUSSION

This research will contribute to a better understanding of participant recruitment to RCTs, which could enhance efficiency and reduce the waste of resources in clinical research with a comprehensive, concerted, international effort.

Developing a roadmap to improve trial delivery for under-served groups: results from a UK multi-stakeholder process

Background

Participants in clinical research studies often do not reflect the populations for which healthcare interventions are needed or will be used. Enhancing representation of under-served groups in clinical research is important to ensure that research findings are widely applicable. We describe a multicomponent workstream project to improve representation of under-served groups in clinical trials.

Methods

The project comprised three main strands: (1) a targeted scoping review of literature to identify previous work characterising under-served groups and barriers to inclusion, (2) surveys of professional stakeholders and participant representative groups involved in research delivery to refine these initial findings and identify examples of innovation and good practice and (3) a series of workshops bringing together key stakeholders from funding, design, delivery and participant groups to reach consensus on definitions, barriers and a strategic roadmap for future work. The work was commissioned by the UK National Institute for Health Research Clinical Research Network. Output from these strands was integrated by a steering committee to generate a series of goals, workstream plans and a strategic roadmap for future development work in this area.

Results

‘Under-served groups’ was identified and agreed by the stakeholder group as the preferred term. Three-quarters of stakeholders felt that a clear definition of under-served groups did not currently exist; definition was challenging and context-specific, but exemplar groups (e.g. those with language barriers or mental illness) were identified as under-served. Barriers to successful inclusion of under-served groups could be clustered into communication between research teams and participant groups; how trials are designed and delivered, differing agendas of research teams and participant groups; and lack of trust in the research process. Four key goals for future work were identified: building long-term relationships with under-served groups, developing training resources to improve design and delivery of trials for under-served groups, developing infrastructure and systems to support this work and working with funders, regulators and other stakeholders to remove barriers to inclusion.

Conclusions

The work of the INCLUDE group over the next 12 months will build on these findings by generating resources customised for different under-served groups to improve the representativeness of trial populations.

Two-by-two factorial randomised study within a trial (SWAT) to evaluate strategies for follow-up in a randomised prevention trial

BACKGROUND

Failure to collect outcome data in randomised trials can result in bias and loss of statistical power. Further evaluations of strategies to increase retention are required. We assessed the effectiveness of two strategies for retention in a randomised prevention trial using a two-by-two factorial randomised study within a trial (SWAT).

METHODS

Parents of babies included in the host trial were randomised to (1) short message service (SMS) notification prior to sending questionnaires at 3, 6, 12 and 18 months versus no SMS notification and (2) a £10 voucher sent with the invitation letter for the primary follow-up visit at 24 months or given at the visit. The two co-primary outcomes were collection of host trial (1) questionnaire data at interim follow-up times and (2) primary outcome at 24 months during a home/clinic visit with a research nurse.

RESULTS

Between November 2014 and November 2016, 1394 participants were randomised: 350 to no SMS + voucher at visit, 345 to SMS + voucher at visit, 352 to no SMS + voucher before visit and 347 to SMS + voucher before visit. Overall questionnaire data was collected at interim follow-up times for 75% in both the group allocated to the prior SMS notification and the group allocated to no SMS notification (odds ratio (OR) SMS versus none 1.02, 95% CI 0.83 to 1.25). Host trial primary outcome data was collected at a visit for 557 (80%) allocated to the voucher before the visit in the invitation letter and for 566 (81%) whose parents were allocated to receive the voucher at the visit (OR before versus at visit 0.89, 95% CI 0.69 to 1.17).

CONCLUSION

There was no evidence of a difference in retention according to SMS notification or voucher timing. Future synthesis of SWAT results is required to be able to detect small but important incremental effects of retention strategies.

TRIAL REGISTRATION

ISRCTN registry, ID: ISRCTN21528841. Registered on 25 July 2014. SWAT Repository Store ID 25.

Using digital tools in the recruitment and retention in randomised controlled trials: survey of UK Clinical Trial Units and a qualitative study

BACKGROUND

Recruitment and retention of participants in randomised controlled trials (RCTs) is a key determinant of success but is challenging. Trialists and UK Clinical Research Collaboration (UKCRC) Clinical Trials Units (CTUs) are increasingly exploring the use of digital tools to identify, recruit and retain participants. The aim of this UK National Institute for Health Research (NIHR) study was to identify what digital tools are currently used by CTUs and understand the performance characteristics required to be judged useful.

METHODS

A scoping of searches (and a survey with NIHR funding staff), a survey with all 52 UKCRC CTUs and 16 qualitative interviews were conducted with five stakeholder groups including trialists within CTUs, funders and research participants. A purposive sampling approach was used to conduct the qualitative interviews during March-June 2018. Qualitative data were analysed using a content analysis and inductive approach.

RESULTS

Responses from 24 (46%) CTUs identified that database-screening tools were the most widely used digital tool for recruitment, with the majority being considered effective. The reason (and to whom) these tools were considered effective was in identifying potential participants (for both Site staff and CTU staff) and reaching recruitment target (for CTU staff/CI). Fewer retention tools were used, with short message service (SMS) or email reminders to participants being the most reported. The qualitative interviews revealed five themes across all groups: 'security and transparency'; 'inclusivity and engagement'; 'human interaction'; 'obstacles and risks'; and 'potential benefits'. There was a high level of stakeholder acceptance of the use of digital tools to support trials, despite the lack of evidence to support them over more traditional techniques. Certain differences and similarities between stakeholder groups demonstrated the complexity and challenges of using digital tools for recruiting and retaining research participants.

CONCLUSIONS

Our studies identified a range of digital tools in use in recruitment and retention of RCTs, despite the lack of high-quality evidence to support their use. Understanding the type of digital tools in use to support recruitment and retention will help to inform funders and the wider research community about their value and relevance for future RCTs. Consideration of further focused digital tool reviews and primary research will help to reduce gaps in the evidence base.

Generating comparative evidence on new drugs and devices before approval

Fewer than half of new drugs have data on their comparative benefits and harms against existing treatment options at the time of regulatory approval in Europe and the USA. Even when active-comparator trials exist, they might not produce meaningful data to inform decisions in clinical practice and health policy. The uncertainty associated with the paucity of well designed active-comparator trials has been compounded by legal and regulatory changes in Europe and the USA that have created a complex mix of expedited programmes aimed at facilitating faster access to new drugs. Comparative evidence generation is even sparser for medical devices. Some have argued that the current process for regulatory approval needs to generate more evidence that is useful for patients, clinicians, and payers in health-care systems. We propose a set of five key principles relevant to the European Medicines Agency, European medical device regulatory agencies, US Food and Drug Administration, as well as payers, that we believe will provide the necessary incentives for pharmaceutical and device companies to generate comparative data on drugs and devices and assure timely availability of evidence that is useful for decision making. First, labelling should routinely inform patients and clinicians whether comparative data exist on new products. Second, regulators should be more selective in their use of programmes that facilitate drug and device approvals on the basis of incomplete benefit and harm data. Third, regulators should encourage the conduct of randomised trials with active comparators. Fourth, regulators should use prospectively designed network meta-analyses based on existing and future randomised trials. Last, payers should use their policy levers and negotiating power to incentivise the generation of comparative evidence on new and existing drugs and devices, for example, by explicitly considering proven added benefit in pricing and payment decisions.

Online randomized controlled experiments at scale: lessons and extensions to medicine

Background

Many technology companies, including Airbnb, Amazon, Booking.com, eBay, Facebook, Google, LinkedIn, Lyft, Microsoft, Netflix, Twitter, Uber, and Yahoo!/Oath, run online randomized controlled experiments at scale, namely hundreds of concurrent controlled experiments on millions of users each, commonly referred to as A/B tests. Originally derived from the same statistical roots, randomized controlled trials (RCTs) in medicine are now criticized for being expensive and difficult, while in technology, the marginal cost of such experiments is approaching zero and the value for data-driven decision-making is broadly recognized.

Methods and results

This is an overview of key scaling lessons learned in the technology field. They include (1) a focus on metrics, an overall evaluation criterion and thousands of metrics for insights and debugging, automatically computed for every experiment; (2) quick release cycles with automated ramp-up and shut-down that afford agile and safe experimentation, leading to consistent incremental progress over time; and (3) a culture of ‘test everything’ because most ideas fail and tiny changes sometimes show surprising outcomes worth millions of dollars annually.

Technological advances, online interactions, and the availability of large-scale data allowed technology companies to take the science of RCTs and use them as online randomized controlled experiments at large scale with hundreds of such concurrent experiments running on any given day on a wide range of software products, be they web sites, mobile applications, or desktop applications. Rather than hindering innovation, these experiments enabled accelerated innovation with clear improvements to key metrics, including user experience and revenue. As healthcare increases interactions with patients utilizing these modern channels of web sites and digital health applications, many of the lessons apply. The most innovative technological field has recognized that systematic series of randomized trials with numerous failures of the most promising ideas leads to sustainable improvement.

Conclusion

While there are many differences between technology and medicine, it is worth considering whether and how similar designs can be applied via simple RCTs that focus on healthcare decision-making or service delivery. Changes – small and large – should undergo continuous and repeated evaluations in randomized trials and learning from their results will enable accelerated healthcare improvements.

Trial Forge Guidance 2: how to decide if a further Study Within A Trial (SWAT) is needed

The evidence base available to trialists to support trial process decisions—e.g. how best to recruit and retain participants, how to collect data or how to share the results with participants—is thin. One way to fill gaps in evidence is to run Studies Within A Trial, or SWATs. These are self-contained research studies embedded within a host trial that aim to evaluate or explore alternative ways of delivering or organising a particular trial process.

SWATs are increasingly being supported by funders and considered by trialists, especially in the UK and Ireland. At some point, increasing SWAT evidence will lead funders and trialists to ask: given the current body of evidence for a SWAT, do we need a further evaluation in another host trial? A framework for answering such a question is needed to avoid SWATs themselves contributing to research waste.

This paper presents criteria on when enough evidence is available for SWATs that use randomised allocation to compare different interventions.

Comparison of randomized controlled trials discontinued or revised for poor recruitment and completed trials with the same research question: a matched qualitative study

Background

More than a quarter of randomized controlled trials (RCTs) are prematurely discontinued, mostly due to poor recruitment of patients. In this study, we systematically compared RCTs discontinued or revised for poor recruitment and completed RCTs with the same underlying research question to better understand the causes of poor recruitment, particularly related to methodological aspects and context-specific study settings.

Methods

We compared RCTs that were discontinued or revised for poor recruitment to RCTs that were completed as planned, matching in terms of population and intervention. Based on an existing sample of RCTs discontinued or revised due to poor recruitment, we identified matching RCTs through a literature search for systematic reviews that cited the discontinued or revised RCT and matching completed RCTs without poor recruitment. Based on extracted data, we explored differences in the design, conduct, and study settings between RCTs with and without poor recruitment, separately for each research question using semi-structured discussions.

Results

We identified 15 separate research questions with a total of 29 RCTs discontinued or revised for poor recruitment and 48 RCTs completed as planned. Prominent research areas in the sample were cancer and acute care. The mean number of RCTs with poor recruitment per research question was 1.9 ranging from 1 to 4 suggesting clusters of research questions or settings prone to recruitment problems. The reporting quality of the recruitment process in RCT publications was generally low. We found that RCTs with poor recruitment often had narrower eligibility criteria, were investigator- rather than industry-sponsored, were associated with a higher burden for patients and recruiters, sometimes used outdated control interventions, and were often launched later in time than RCTs without poor recruitment compromising uncertainty about tested interventions through emerging evidence. Whether a multi- or single-center setting was advantageous for patient recruitment seemed to depend on the research context.

Conclusions

Our study confirmed previously identified causes for poor recruitment, i.e., narrow eligibility criteria, investigator sponsorship, and a reduced motivation of patients and recruiters. Newly identified aspects were that researchers need to be aware of all other RCTs on a research question so that compromising effects on the recruitment can be minimized and that a larger number of centers is not always advantageous.

Don’t abandon RCTs in IVF. We don’t even understand them

The conclusion of the Human Fertilisation and Embryology Authority that ‘add-on’ therapies in IVF are not supported by high-quality evidence has prompted new questions regarding the role of the randomized controlled trial (RCT) in evaluating infertility treatments. Critics argue that trials are cumbersome tools that provide irrelevant answers. Instead, they argue that greater emphasis should be placed on large observational databases, which can be analysed using powerful algorithms to determine which treatments work and for whom. Although the validity of these arguments rests upon the sciences of statistics and epidemiology, the discussion to date has largely been conducted without reference to these fields. We aim to remedy this omission, by evaluating the arguments against RCTs in IVF from a primarily methodological perspective. We suggest that, while criticism of the status quo is warranted, a retreat from RCTs is more likely to make things worse for patients and clinicians.

Design, planning and implementation lessons learnt from a surgical multi-centre randomised controlled trial

Background

Increasingly, pragmatic randomised controlled trials are being used to evaluate surgical interventions, although they present particular difficulties in regards to recruitment and retention.

Methods

Procedures and processes related to implementation of a multi-centre pragmatic surgical randomised controlled trial are discussed. In this surgical trial, forecasting of consent rates based on similar trials and micro-costing of study activities with research partners were undertaken and a video was produced targeting recruiting staff with the aim of aiding recruitment. The baseline assessments were reviewed to ensure the timing did not impact on the outcome. Attrition due to procedure waiting time was monitored and data were triangulated for the primary outcome to ensure adequate follow-up data.

Results

Forecasting and costing ensured that the recruitment window was of adequate length and adequate resource was available for study procedures at multiple clinics in each hospital. Recruiting staff found the recruitment video useful. The comparison of patient-reported data collected prior to randomisation and prior to treatment provided confidence in the baseline data. Knowledge of participant dropout due to delays in treatment meant we were able to increase the recruitment target in a timely fashion, and along with the triangulation of data sources, this ensured adequate follow-up of randomised participants.

Conclusions

This paper provides a range of evidence-based and experience-based approaches which, collectively, resulted in meeting our study objectives and from which lessons may be transferable.

Trial registration

ISRCTN, ISRCTN41394716. Registered on 10 May 2012.

UKCRN Study ID: 12486.

Identifying important barriers to recruitment of patients in randomised clinical studies using a questionnaire for study personnel

Background

Many randomised controlled trials (RCT) fail to meet their recruitment goals. Study personnel play a key role in recruitment. The aim of this study was to identify successful strategies that study personnel consider to be important in patient recruitment to RCT.

Methods

We constructed a questionnaire based on the literature, discussions with colleagues and our own experience as trialists. The survey was named “What is Important for Making a Study Successful questionnaire” (WIMSS-q). Our target group was the study personnel in the ongoing EFFECTS study. The questionnaire was sent out electronically to all physicians and nurses (n = 148). Success factors and barriers were divided according to patient, centre and study level, respectively.

Results

Responses were received from 94% of the study personnel (139/148). The five most important factors at centre level for enhancing recruitment were that the research question was important (97%), a simple procedure for providing information and gaining consent (92%), a highly engaged local principal investigator and research nurse (both 87%), and that study-related follow-ups are practically feasible and possible to coordinate with the clinical follow-up (87%). The most significant barrier at the local centre was lack of time and resources devoted to research (72%). Important patient-related barriers were fear of side effects (35%) and language problems (30%).

Conclusions

For recruitment in an RCT to be successful, the research question must be relevant, and the protocol must be simple and easy to implement in the daily routine.

Trial registration

The protocol for this study was registered at the Northern Ireland Hub for trials methodology research (SWAT ID 64). The EFFECTS study has EudraCT number 2011–006130-16 and was registered 17 February 2016 at ClinicalTrials.gov number NCT02683213.

A high-volume, low-cost approach to participant screening and enrolment: Experiences from the T4DM diabetes prevention trial

BACKGROUND/AIMS

Participant recruitment to diabetes prevention randomised controlled trials is challenging and expensive. The T4DM study, a multicentre, Australia-based, Phase IIIb randomised controlled trial of testosterone to prevent Type 2 diabetes in men aged 50-74 years, faced the challenge of screening a large number of prospective participants at a small number of sites, with few staff, and a limited budget for screening activities. This article evaluates a high-volume, low-cost, semi-automated approach to screen and enrol T4DM study participants.

METHODS

We developed a sequential multi-step screening process: (1) web-based pre-screening, (2) laboratory screening through a network of third-party pathology centres, and (3) final on-site screening, using online data collection, computer-driven eligibility checking, and automated, email-based communication with prospective participants. Phone- and mail-based data collection and communication options were available to participants at their request. The screening process was administered by the central coordinating centre through a central data management system.

RESULTS

Screening activities required staffing of approximately 1.6 full-time equivalents over 4 years. Of 19,022 participants pre-screened, 13,108 attended a third-party pathology collection centre for laboratory screening, 1217 received final, on-site screening, and 1007 were randomised. In total, 95% of the participants opted for online pre-screening over phone-based pre-screening. Screening costs, including both direct and staffing costs, totalled AUD1,420,909 (AUD75 per subject screened and AUD1411 per randomised participant).

CONCLUSION

A multi-step, semi-automated screening process with web-based pre-screening facilitated low-cost, high-volume participant enrolment to this large, multicentre randomised controlled trial. Centralisation and automation of screening activities resulted in substantial savings compared to previous, similar studies. Our screening approach could be adapted to other randomised controlled trial settings to minimise the cost of screening large numbers of participants.

Rapid Analgesia for Prehospital hip Disruption (RAPID): findings from a randomised feasibility study

Background

In managing hip fracture, effective pain relief before admission to hospital is difficult without risking side effects. Although emergency departments routinely use fascia iliaca compartment block (FICB), there has been little evaluation of its use by paramedics before hospital admission. We aimed to assess whether a multi-centre randomised trial to evaluate FICB was feasible.

Methods

Volunteer paramedics used scratchcards to allocate patients with hip fracture at random between FICB and pain relief as usual. Primary outcomes were mortality and quality of life. We also measured adverse events, costs, final diagnosis, length of stay in hospital, pain scores and quality of care and collected qualitative data about acceptability to patients in interviews, and paramedics in focus groups. We pre-specified criteria for deciding whether to progress to a fully powered trial based on the recruitment of paramedics and patients, delivery of FICB, retrieval of outcome data, safety, acceptability, and diagnostic accuracy of hip fracture.

Results

We effectively met all progression criteria: we recruited 19 paramedics who randomly allocated 71 patients between trial arms between 28 June 2016 and 31 July 2017; 57 (31 experimental arm, 26 usual care arm, 80% overall) retrospectively consented to follow-up. Just over half (17/31) of experimental participants received FICB; all others had contraindications, including nine taking anticoagulants. Four of the 31 participants assigned FICB and six of the 26 assigned usual care died within 6 months of hospital admission. Serious adverse events were also similar: 3/35 experimental versus 4/36 in usual care. Paramedics’ recognition of hip fracture had sensitivity of 49/64 (77%) with a positive predictive value of 46/57 (81%). We received quality of life questionnaires for 30 of 49 patients (61%) at 1 month and 12 of 17 (71%) at 6 months. Patient satisfaction was similar: experimental mean 3.4 (n = 20) versus 3.5 (n = 13) for usual care.

Conclusions

RAPID met all progression criteria within reasonable limits. As equipoise remains, we plan to undertake a fully powered multi-centre trial to test clinical and cost effectiveness of paramedic-administered FICB at the scene of hip fracture.

Trial registration

ISRCTN 60065373 sought 5 November 2015.

Electronic supplementary material

The online version of this article (10.1186/s40814-019-0454-1) contains supplementary material, which is available to authorized users.

A systematic review of feasibility studies promoting the use of mobile technologies in clinical research

Mobile technologies, such as smart phone applications, wearables, ingestibles, and implantables, are increasingly used in clinical research to capture study endpoints. On behalf of the Clinical Trials Transformation Initiative, we aimed to conduct a systematic scoping review and compile a database summarizing pilot studies addressing mobile technology sensor performance, algorithm development, software performance, and/or operational feasibility, in order to provide a resource for guiding decisions about which technology is most suitable for a particular trial. Our systematic search identified 275 publications meeting inclusion criteria. From these papers, we extracted data including the medical condition, concept of interest captured by the mobile technology, outcomes captured by the digital measurement, and details regarding the sensors, algorithms, and study sample. Sixty-seven percent of the technologies identified were wearable sensors, with the remainder including tablets, smartphones, implanted sensors, and cameras. We noted substantial variability in terms of reporting completeness and terminology used. The data have been compiled into an online database maintained by the Clinical Trials Transformation Initiative that can be filtered and searched electronically, enabling a user to find information most relevant to their work. Our long-term goal is to maintain and update the online database, in order to promote standardization of methods and reporting, encourage collaboration, and avoid redundant studies, thereby contributing to the design and implementation of efficient, high-quality trials.

Abdominal massage plus advice, compared with advice only, for neurogenic bowel dysfunction in MS: a RCT

BACKGROUND

Between 50% and 80% of people with multiple sclerosis (PwMS) experience neurogenic bowel dysfunction (NBD) (i.e. constipation and faecal incontinence) that affects quality of life and can lead to hospitalisation.

OBJECTIVES

To determine the clinical effectiveness and cost-effectiveness of abdominal massage plus advice on bowel symptoms on PwMS compared with advice only. A process evaluation investigated the factors that affected the clinical effectiveness and possible implementation of the different treatments.

DESIGN

A randomised controlled trial with process evaluation and health economic components. Outcome analysis was undertaken blind.

SETTING

The trial took place in 12 UK hospitals.

PARTICIPANTS

PwMS who had 'bothersome' NBD.

INTERVENTION

Following individualised training, abdominal massage was undertaken daily for 6 weeks (intervention group). Advice on good bowel management as per the Multiple Sclerosis Society advice booklet was provided to both groups. All participants received weekly telephone calls from the research nurse.

MAIN OUTCOME MEASURES

The primary outcome was the difference between the intervention and control groups in change in the NBD score from baseline to week 24. Secondary outcomes were measured via a bowel diary, adherence diary, the Constipation Scoring System, patient resource questionnaire and the EuroQol-5 Dimensions, five-level version (EQ-5D-5L).

RESULTS

A total of 191 participants were finalised, 189 of whom were randomised (two participants were finalised in error) (control group, n = 99; intervention group, n = 90) and an intention-to-treat analysis was performed. The mean age was 52 years (standard deviation 10.83 years), 81% (n = 154) were female and 11% (n = 21) were wheelchair dependent. Fifteen participants from the intervention group and five from the control group were lost to follow-up. The change in NBD score by week 24 demonstrated no significant difference between groups [mean difference total score -1.64, 95% confidence interval (CI) -3.32 to 0.04; p = 0.0558]; there was a significant difference between groups in the change in the frequency of stool evacuation per week (mean difference 0.62, 95% CI 0.03 to 1.21; p = 0.039) and in the number of times per week that participants felt that they emptied their bowels completely (mean difference 1.08, 95% CI 0.41 to 1.76; p = 0.002), in favour of the intervention group. Of participant interviewees, 75% reported benefits, for example less difficulty passing stool, more complete evacuations, less bloated, improved appetite, and 85% continued with the massage. A cost-utility analysis conducted from a NHS and patient cost perspective found in the imputed sample with bootstrapping a mean incremental outcome effect of the intervention relative to usual care of -0.002 quality-adjusted life-years (QALYs) (95% CI -0.029 to 0.027 QALYs). In the same imputed sample with bootstrapping, the mean incremental cost effect of the intervention relative to usual care was £56.50 (95% CI -£372.62 to £415.68). No adverse events were reported. Limitations include unequal randomisation, dropout and the possibility of ineffective massage technique.

CONCLUSION

The increment in the primary outcome favoured the intervention group, but it was small and not statistically significant. The economic analysis identified that the intervention was dominated by the control group. Given the small improvement in the primary outcome, but not in terms of QALYs, a low-cost version of the intervention might be considered worthwhile by some patients.

FUTURE WORK

Research is required to establish possible mechanisms of action and modes of massage delivery.

TRIAL REGISTRATION

Current Controlled Trials ISRCTN85007023 and NCT03166007.

FUNDING

This project was funded by the National Institute for Health Research Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 22, No. 58. See the NIHR Journals Library website for further project information.

Evaluating recruitment strategies for AUSPICE, a large Australian community-based randomised controlled trial

OBJECTIVES

To examine the effectiveness of different strategies for recruiting participants for a large Australian randomised controlled trial (RCT), the Australian Study for the Prevention through Immunisation of Cardiovascular Events (AUSPICE).

DESIGN, SETTING, PARTICIPANTS

Men and women aged 55-60 years with at least two cardiovascular risk factors (hypertension, hypercholesterolaemia, overweight/obesity) were recruited for a multicentre placebo-controlled RCT assessing the effectiveness of 23-valent pneumococcal polysaccharide vaccine (23vPPV) for preventing cardiovascular events.

METHODS

Invitations were mailed by the Australian Department of Human Services to people in the Medicare database aged 55-60 years; reminders were sent 2 weeks later. Invitees could respond in hard copy or electronically. Direct recruitment was supplemented by asking invitees to extend the invitation to friends and family (snowball sampling) and by Facebook advertising.

MAIN OUTCOME

Proportions of invitees completing screening questionnaire and recruited for participation in the RCT.

RESULTS

21 526 of 154 992 invited people (14%) responded by completing the screening questionnaire, of whom 4725 people were eligible and recruited for the study. Despite the minimal study burden (one questionnaire, one clinic visit), the overall participation rate was 3%, or an estimated 10% of eligible persons. Only 16% of eventual participants had responded within 2 weeks of the initial invitation letter (early responders); early and late responders did not differ in their demographic or medical characteristics. Socio-economic disadvantage did not markedly influence response rates. Facebook advertising and snowball sampling did not increase recruitment.

CONCLUSIONS

Trial participation rates are low, and multiple concurrent methods are needed to maximise recruitment. Social media strategies may not be successful in older age groups.

TRIAL REGISTRATION

Australian New Zealand Clinical Trials Registry, ACTRN12615000536561.

The QuinteT Recruitment Intervention supported five randomized trials to recruit to target: a mixed-methods evaluation

OBJECTIVE

To evaluate the impact of the QuinteT Recruitment Intervention (QRI) on recruitment in challenging randomized controlled trials (RCTs) that have applied the intervention. The QRI aims to understand recruitment difficulties and then implements "QRI actions" to address these as recruitment proceeds.

STUDY DESIGN AND SETTING

A mixed-methods study, comprising (1) before-and-after comparisons of recruitment rates and the numbers of patients approached and (2) qualitative case studies, including documentary analysis and interviews with RCT investigators.

RESULTS

Five UK-based publicly funded RCTs were included in the evaluation. All recruited to target. Randomized controlled trial 2 and RCT 5 both received up-front prerecruitment training before the intervention was applied. Randomized controlled trial 2 did not encounter recruitment issues and recruited above target from its outset. Recruitment difficulties, particularly communication issues, were identified and addressed through QRI actions in RCTs 1, 3, 4, and 5. Randomization rates significantly improved after QRI action in RCTs 1, 3, and 4. Quintet Recruitment Intervention actions addressed issues with approaching eligible patients in RCTs 3 and 5, which both saw significant increases in the number of patients approached. Trial investigators reported that the QRI had unearthed issues they had been unaware of and reportedly changed their practices after QRI action.

CONCLUSION

There is promising evidence to suggest that the QRI can support recruitment to difficult RCTs. This needs to be substantiated with future controlled evaluations.

Impact of patient and public involvement on enrolment and retention in clinical trials: systematic review and meta-analysis

OBJECTIVE

To investigate the impact of patient and public involvement (PPI) on rates of enrolment and retention in clinical trials and explore how this varies with the context and nature of PPI.

DESIGN

Systematic review and meta-analysis.

DATA SOURCES

Ten electronic databases, including Medline, INVOLVE Evidence Library, and clinical trial registries.

ELIGIBILITY CRITERIA

Experimental and observational studies quantitatively evaluating the impact of a PPI intervention, compared with no intervention or non-PPI intervention(s), on participant enrolment and/or retention rates in a clinical trial or trials. PPI interventions could include additional non-PPI components inseparable from the PPI (for example, other stakeholder involvement).

DATA EXTRACTION AND ANALYSIS

Two independent reviewers extracted data on enrolment and retention rates, as well as on the context and characteristics of PPI intervention, and assessed risk of bias. Random effects meta-analyses were used to determine the average effect of PPI interventions on enrolment and retention in clinical trials: main analysis including randomised studies only, secondary analysis adding non-randomised studies, and several exploratory subgroup and sensitivity analyses.

RESULTS

26 studies were included in the review; 19 were eligible for enrolment meta-analysis and five for retention meta-analysis. Various PPI interventions were identified with different degrees of involvement, different numbers and types of people involved, and input at different stages of the trial process. On average, PPI interventions modestly but significantly increased the odds of participant enrolment in the main analysis (odds ratio 1.16, 95% confidence interval and prediction interval 1.01 to 1.34). Non-PPI components of interventions may have contributed to this effect. In exploratory subgroup analyses, the involvement of people with lived experience of the condition under study was significantly associated with improved enrolment (odds ratio 3.14 v 1.07; P=0.02). The findings for retention were inconclusive owing to the paucity of eligible studies (odds ratio 1.16, 95% confidence interval 0.33 to 4.14), for main analysis).

CONCLUSIONS

These findings add weight to the case for PPI in clinical trials by indicating that it is likely to improve enrolment of participants, especially if it includes people with lived experience of the health condition under study. Further research is needed to assess which types of PPI work best in particular contexts, the cost effectiveness of PPI, the impact of PPI at earlier stages of trial design, and the impact of PPI interventions specifically targeting retention.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42016043808.

Partially systematic thoughts on the history of systematic reviews

Six years after the launch of Systematic Reviews by Biomed Central, this article is part of the celebration of the journal. It contains personal reflections on the past, present and future of systematic reviews, using examples relevant to the role of systematic reviews in cataloguing and analysing research, assessing quality and planning new studies. The focus is on the most common of the various types of systematic review in health and social care, namely those assessing the effects of interventions.

Effect of two different participant information sheets on recruitment to a falls trial: An embedded randomised recruitment trial

BACKGROUND/AIMS

Recruitment to trials of intervention for older people who fall is challenging. Evidence suggests that the word falls has negative connotations for older people, and this may present a barrier to engaging with trials in this area. We therefore tested whether a participant information sheet that minimised reference to falls could improve recruitment rates.

METHODS

We conducted a study within a trial, embedded within a randomised controlled trial of vitamin K versus placebo to improve postural sway in patients aged 65 and over with a history of falls. Potential participants were identified from primary care lists in 14 practices and were randomised to receive either a standard participant information sheet or an information sheet minimising use of the word falls, instead focussing on maintenance of health, fitness and balance. The primary outcome for this embedded trial was the proportion of responses expressing interest in participating received in each arm. Secondary outcomes were the proportion of those contacted attending a screening visit, consenting at screening, and the proportion contacted who were randomised into the main trial.

RESULTS

In all, 4145 invitations were sent, with an overall response rate of 444 (10.7%). In all, 2148 individuals received the new information sheet (minimising reference to falls); 1997 received the standard information sheet. There was no statistically significant difference in response rate between those individuals sent the new information sheet and those sent the standard information sheet (10.1% vs 11.4%; difference 1.3% (95% confidence interval -0.6% to 3.2%); p = 0.19). Similarly, we found no statistically significant difference between the percentage of those who attended and consented at screening in the two groups (2.1% vs 2.7%; difference 0.6% (95% confidence interval: -0.4% to 1.6%); p = 0.20), and no statistically significant difference between the percentage randomised in the two groups (2.0% vs 2.6%; difference 0.6% (95% confidence interval -0.4% to 1.6%); p = 0.20).

CONCLUSIONS

Use of a participant information sheet minimising reference to falls did not lead to a greater response rate in this trial targeting older people with a history of falls.

Doing trials within trials: a qualitative study of stakeholder views on barriers and facilitators to the routine adoption of methodology research in clinical trials

Background

Randomised controlled trials are the cornerstone of evidence-based health care, yet many trials struggle with recruitment and retention. All too often the methodologies employed to address these problems are not evidence-based, as rigorous methodological research on these issues is rare. The current research sought to identify barriers to the routine implementation of methodology research around recruitment and retention.

Methods

All registered UK clinical trials unit directors were sent a short questionnaire and invited to interview. Representatives of funding bodies and other stakeholders were also approached. Interviews were recorded and the content analysed.

Results

Data were grouped into four themes: acceptance of the need for methodological research; trial funding and development; trial processes; and organisational factors. The need to improve the evidence base for trials methodology is well established, but numerous barriers to implementation were perceived.

Conclusions

The knowledge and expertise required to routinely implement methodology research exists within the current research structures, and there are clear opportunities to develop the evidence base. However, for this to be achieved there is also a need for clear strategic coordination within the sector and promotion of the necessary resources.

SMOOTH: Self-Management of Open Online Trials in Health analysis found improvements were needed for reporting methods of internet-based trials

BACKGROUND AND OBJECTIVES

The growth of trials conducted over the internet has increased, but with little practical guidance for their conduct, and it is sometimes challenging for researchers to adapt the conventions used in face-to-face trials and maintain the validity of the work. The aim of the study is to systematically explore existing self-recruited online randomized trials of self-management interventions and analyze the trials to assess their strengths and weaknesses, the quality of reporting, and the involvement of lay persons as collaborators in the research process.

STUDY DESIGN AND SETTINGS

The Online Randomized Controlled Trials of Health Information Database was used as the sampling frame to identify a subset of self-recruited online trials of self-management interventions. The authors cataloged what these online trials were assessing, appraised study quality, extracted information on how trials were run, and assessed the potential for bias. We searched out how public and patient participation was integrated into online trial design and how this was reported. We recorded patterns of use for registration, reporting, settings, informed consent, public involvement, supplementary materials, and dissemination planning.

RESULTS

The sample included 41 online trials published from 2002 to 2015. The barriers to replicability and risk of bias in online trials included inadequate reporting of blinding in 28/41 (68%) studies; high attrition rates with incomplete or unreported data in 30/41 (73%) of trials; and 26/41 (63%) of studies were at high risk for selection bias as trial registrations were unreported. The methods for (23/41, 56%) trials contained insufficient information to replicate the trial, 19/41 did not report piloting the intervention. Only 2/41 studies were cross-platform compatible. Public involvement was most common for advisory roles (n = 9, 22%), and in the design, usability testing, and piloting of user materials (n = 9, 22%).

CONCLUSION

This study catalogs the state of online trials of self-management in the early 21st century and provides insights for online trials development as early as the protocol planning stage. Reporting of trials was generally poor and, in addition to recommending that authors report their trials in accordance with CONSORT guidelines, we make recommendations for researchers writing protocols, reporting on and evaluating online trials. The research highlights considerable room for improvement in trial registration, reporting of methods, data management plans, and public and patient involvement in self-recruited online trials of self-management interventions.

Automated recruitment and randomisation for an efficient randomised controlled trial in primary care

BACKGROUND/AIMS

Use of electronic health records and information technology to deliver more efficient clinical trials is attracting the attention of research funders and researchers. We report on methodological issues and data quality for a comparison of 'automated' and manual (or 'in-practice') methods for recruitment and randomisation in a large randomised controlled trial, with individual patient allocation in primary care.

METHODS

We conducted a three-arm randomised controlled trial in primary care to evaluate interventions to improve the uptake of invited NHS health checks for cardiovascular risk assessment. Eligible participants were identified using a borough-wide health check management information system. An in-practice recruitment and randomisation method used at 12 general practices required the research team to complete monthly visits to each general practice. For the fully automated method, employed for six general practices, randomisation of eligible participants was performed automatically and remotely using a bespoke algorithm embedded in the health check management information system.

RESULTS

There were 8588 and 4093 participants recruited for the manual and automated methods, respectively. The in-practice method was ready for implementation 3 months sooner than the automated method and the in-practice method allowed for full control and documentation of the randomisation procedure. However the in-practice approach was labour intensive and the requirement for participant records to be stored locally resulted in the loss of data for 10 practice months. No records for participants allocated using the automated method were lost. A fixed-effects meta-analysis showed that effect estimates for the primary outcome were consistent for the two allocation methods.

CONCLUSIONS

This trial demonstrated the feasibility of automated recruitment and randomisation methods into a randomised controlled trial performed in primary care. Future research should explore the application of these techniques in other clinical contexts and health care settings.

TRIAL REGISTRATION

Current Controlled Trials, ID: ISRCTN42856343 . Registered on 21 March 2013.

COSCA (Core Outcome Set for Cardiac Arrest) in Adults: An Advisory Statement From the International Liaison Committee on Resuscitation

Cardiac arrest effectiveness trials have traditionally reported outcomes that focus on survival. A lack of consistency in outcome reporting between trials limits the opportunities to pool results for meta-analysis. The COSCA initiative (Core Outcome Set for Cardiac Arrest), a partnership between patients, their partners, clinicians, research scientists, and the International Liaison Committee on Resuscitation, sought to develop a consensus core outcome set for cardiac arrest for effectiveness trials. Core outcome sets are primarily intended for large, randomized clinical effectiveness trials (sometimes referred to as pragmatic trials or phase III/IV trials) rather than for pilot or efficacy studies. A systematic review of the literature combined with qualitative interviews among cardiac arrest survivors was used to generate a list of potential outcome domains. This list was prioritized through a Delphi process, which involved clinicians, patients, and their relatives/partners. An international advisory panel narrowed these down to 3 core domains by debate that led to consensus. The writing group refined recommendations for when these outcomes should be measured and further characterized relevant measurement tools. Consensus emerged that a core outcome set for reporting on effectiveness studies of cardiac arrest (COSCA) in adults should include survival, neurological function, and health-related quality of life. This should be reported as survival status and modified Rankin scale score at hospital discharge, at 30 days, or both. Health-related quality of life should be measured with ≥1 tools from Health Utilities Index version 3, Short-Form 36-Item Health Survey, and EuroQol 5D-5L at 90 days and at periodic intervals up to 1 year after cardiac arrest, if resources allow.

Implementation and trial evidence: a plea for fore-thought

In a world where evidence-based practice is see as the foundation of modern healthcare, this paper asks when and how should we be accounting for the input of patients, the public, dental professionals, commissioners and policy-makers in the evidence generation process?

Data-Driven Methods for Advancing Precision Oncology

PURPOSE OF REVIEW

This article discusses the advances, methods, challenges, and future directions of data-driven methods in advancing precision oncology for biomedical research, drug discovery, clinical research, and practice.

RECENT FINDINGS

Precision oncology provides individually tailored cancer treatment by considering an individual's genetic makeup, clinical, environmental, social, and lifestyle information. Challenges include voluminous, heterogeneous, and disparate data generated by different technologies with multiple modalities such as Omics, electronic health records, clinical registries and repositories, medical imaging, demographics, wearables, and sensors. Statistical and machine learning methods have been continuously adapting to the ever-increasing size and complexity of data. Precision Oncology supportive analytics have improved turnaround time in biomarker discovery and time-to-application of new and repurposed drugs. Precision oncology additionally seeks to identify target patient populations based on genomic alterations that are sensitive or resistant to conventional or experimental treatments. Predictive models have been developed for cancer progression and survivorship, drug sensitivity and resistance, and identification of the most suitable combination treatments for individual patient scenarios. In the future, clinical decision support systems need to be revamped to better incorporate knowledge from precision oncology, thus enabling clinical practitioners to provide precision cancer care.

SUMMARY

Open Omics datasets, machine learning algorithms, and predictive models have enabled the advancement of precision oncology. Clinical decision support systems with integrated electronic health record and Omics data are needed to provide data-driven recommendations to assist clinicians in disease prevention, early identification, and individualized treatment. Additionally, as cancer is a constantly evolving disorder, clinical decision systems will need to be continually updated based on more recent knowledge and datasets.

Trial Forge Guidance 1: what is a Study Within A Trial (SWAT)?

Randomised trials are a central component of all evidence-informed health care systems and the evidence coming from them helps to support health care users, health professionals and others to make more informed decisions about treatment. The evidence available to trialists to support decisions on design, conduct and reporting of randomised trials is, however, sparse. Trial Forge is an initiative that aims to increase the evidence base for trial decision-making and in doing so, to improve trial efficiency.One way to fill gaps in evidence is to run Studies Within A Trial, or SWATs. This guidance document provides a brief definition of SWATs, an explanation of why they are important and some practical 'top tips' that come from existing experience of doing SWATs. We hope the guidance will be useful to trialists, methodologists, funders, approvals agencies and others in making clear what a SWAT is, as well as what is involved in doing one.

What are the main inefficiencies in trial conduct: a survey of UKCRC registered clinical trials units in the UK

BACKGROUND

The UK Clinical Research Collaboration (UKCRC) registered Clinical Trials Units (CTUs) Network aims to support high-quality, efficient and sustainable clinical trials research in the UK. To better understand the challenges in efficient trial conduct, and to help prioritise tackling these challenges, we surveyed CTU staff. The aim was to identify important inefficiencies during two key stages of the trial conduct life cycle: (i) from grant award to first participant, (ii) from first participant to reporting of final results.

METHODS

Respondents were asked to list their top three inefficiencies from grant award to recruitment of the first participant, and from recruitment of the first participant to publication of results. Free text space allowed respondents to explain why they thought these were important. The survey was constructed using SurveyMonkey and circulated to the 45 registered CTUs in May 2013. Respondents were asked to name their unit and job title, but were otherwise anonymous. Free-text responses were coded into broad categories.

RESULTS

There were 43 respondents from 25 CTUs. The top inefficiency between grant award and recruitment of first participant was reported as obtaining research and development (R&D) approvals by 23 respondents (53%), contracts by 22 (51%), and other approvals by 13 (30%). The top inefficiency from recruitment of first participant to publication of results was failure to meet recruitment targets, reported by 19 (44%) respondents. A common comment was that this reflected overoptimistic or inaccurate estimates of recruitment at site. Data management, including case report form design and delays in resolving data queries with sites, was reported as an important inefficiency by 11 (26%) respondents, and preparation and submission for publication by 9 (21%).

CONCLUSIONS

Recommendations for improving the efficiency of trial conduct within the CTUs network include: further reducing unnecessary bureaucracy in approvals and contracting; improving training for site staff; realistic recruitment targets and appropriate feasibility; developing training across the network; improving the working relationships between chief investigators and units; encouraging funders to release sufficient funding to allow prompt recruitment of trial staff; and encouraging more research into how to improve the efficiency and quality of trial conduct.

The effect of using social pressure in cover letters to improve retention in a longitudinal health study: an embedded randomised controlled retention trial

Background

Retention of participants in cohort studies is important for validity. One way to promote retention is by sending a persuasive cover letter with surveys. The study aimed to compare the effectiveness of a covering letter containing social pressure with a standard covering letter on retention in a health cohort study. Social pressure involves persuading people to behave in a certain way by the promise that their actions will be made know to others. We implemented a mild form of social pressure, where the recipient was told that information about whether they responded to the current survey would be noted by the research team and printed on future correspondence from the research team to the recipient.

Methods

The design was an embedded randomised controlled retention trial, conducted between July 2015 and April 2016 in Salford, UK. Participants in the host health cohort study were eligible. They received either: (1) a covering letter with two consecutive surveys (sent six and twelve months after recruitment), containing a social pressure intervention; or (2) a matching letter without the social pressure text. The primary outcome was retention in the host study, defined as return of both surveys. Randomisation was computer-generated, with stratification by household size. Participants were blinded to group assignment. Researchers were blinded for outcome ascertainment.

Results

Adults (n = 4447) aged over 65 years, with a long-term condition and enrolled in the host study, were randomly allocated to receive a social pressure covering letter (n = 2223) or control (n = 2224). All 4447 participants were included in the analysis. Both questionnaires were returned by 1577 participants (71%) sent the social pressure letters and 1511 (68%) sent control letters, a risk difference of 3 percentage points (adjusted odds ratio = 1.16 (95% confidence interval = 1.02–1.33)).

Conclusion

A mild form of social pressure made a small but significant improvement in retention of older adults in a health cohort study. Investigation of social pressure across other research contexts and stronger social pressure messages is warranted.

Trial registration

The host cohort study, the Comprehensive Longitudinal Assessment of Salford Integrated Care (CLASSIC) study is associated with the CLASSIC PROTECTs trial, which is registered on the ISRCTN registry. Trial registration number: ISRCTN12286422. Date of registration 19 June 2014.

Electronic supplementary material

The online version of this article (doi:10.1186/s13063-017-2090-5) contains supplementary material, which is available to authorized users.

Research ethics committee decision-making in relation to an efficient neonatal trial

OBJECTIVE

Randomised controlled trials, a gold-standard approach to reduce uncertainties in clinical practice, are growing in cost and are often slow to recruit. We determined whether methodological approaches to facilitate large, efficient clinical trials were acceptable to UK research ethics committees (RECs).

DESIGN

We developed a protocol in collaboration with parents, for a comparative-effectiveness, randomised controlled trial comparing two widely used blood transfusion practices in preterm infants. We incorporated four approaches to improve recruitment and efficiency: (i) point-of-care design using electronic patient records for patient identification, randomisation and data acquisition, (ii) short two-page information sheet; (iii) explicit mention of possible inclusion benefit; (iv) opt-out consent with enrolment as the default. With the support of the UK Health Research Authority, we submitted an identical protocol to 12 UK REC.

SETTING

RECs in the UK.

MAIN OUTCOME

Number of REC granting favourable opinions.

RESULTS

The use of electronic patient records was acceptable to all RECs; one REC raised concerns about the short parent information sheet, 10 about inclusion benefit and 9 about opt-out consent. Following responses to queries, nine RECs granted a favourable final opinion and three rejected the application because they considered the opt-out consent process invalid.

CONCLUSIONS

A majority of RECs in this study consider the use of electronic patient record data, short information sheets, opt-out consent and mention of possible inclusion benefit to be acceptable in neonatal comparative-effectiveness research. We identified a need for guidance for RECs in relation to opt-out consent processes. These methods provide opportunity to facilitate large randomised controlled trials.

Worldwide trends in volume and quality of published protocols of randomized controlled trials

Introduction

Publishing protocols of randomized controlled trials (RCT) facilitates a more detailed description of study rational, design, and related ethical and safety issues, which should promote transparency. Little is known about how the practice of publishing protocols developed over time. Therefore, this study describes the worldwide trends in volume and methodological quality of published RCT protocols.

Methods

A systematic search was performed in PubMed and EMBASE, identifying RCT protocols published over a decade from 1 September 2001. Data were extracted on quality characteristics of RCT protocols. The primary outcome, methodological quality, was assessed by individual methodological characteristics (adequate generation of allocation, concealment of allocation and intention-to-treat analysis). A comparison was made by publication period (First, September 2001- December 2004; Second, January 2005-May 2008; Third, June 2008-September 2011), geographical region and medical specialty.

Results

The number of published RCT protocols increased from 69 in the first, to 390 in the third period (p<0.0001). Internal medicine and paediatrics were the most common specialty topics. Whereas most published RCT protocols in the first period originated from North America (n = 30, 44%), in the second and third period this was Europe (respectively, n = 65, 47% and n = 190, 48%, p = 0.02). Quality of RCT protocols was higher in Europe and Australasia, compared to North America (OR = 0.63, CI = 0.40–0.99, p = 0.04). Adequate generation of allocation improved with time (44%, 58%, 67%, p = 0.001), as did concealment of allocation (38%, 53%, 55%, p = 0.03). Surgical protocols had the highest quality among the three specialty topics used in this study (OR = 1.94, CI = 1.09–3.45, p = 0.02).

Conclusion

Publishing RCT protocols has become popular, with a five-fold increase in the past decade. The quality of published RCT protocols also improved, although variation between geographical regions and across medical specialties was seen. This emphasizes the importance of international standards of comprehensive training in RCT methodology.

The COMET Initiative database: progress and activities update (2015)

This letter describes the substantial activity on the Core Outcome Measure in Effectiveness Trials (COMET) website in 2015, updating our earlier progress reports for the period from the launch of the COMET website and database in August 2011 to December 2014. As in previous years, 2015 saw further increases in the annual number of visits to the website, the number of pages viewed and the number of searches undertaken. The sustained growth in use of the website and database suggests that COMET is continuing to gain interest and prominence, and that the resources are useful to people interested in the development of core outcome sets.