Choosing and evaluating randomisation methods in clinical trials: a qualitative study

BACKGROUND

There exist many different methods of allocating participants to treatment groups during a randomised controlled trial. Although there is research that explores trial characteristics that are associated with the choice of method, there is still a lot of variety in practice not explained. This study used qualitative methods to explore more deeply the motivations behind researchers' choice of randomisation, and which features of the method they use to evaluate the performance of these methods.

METHODS

Data was collected from online focus groups with various stakeholders involved in the randomisation process. Focus groups were recorded and then transcribed verbatim. A thematic analysis was used to analyse the transcripts.

RESULTS

Twenty-five participants from twenty clinical trials units across the UK were recruited to take part in one of four focus groups. Four main themes were identified: how randomisation methods are selected; researchers' opinions of the different methods; which features of the method are desirable and ways to measure method features. Most researchers agree that the randomisation method should be selected based on key trial characteristics; however, for many, a unit standard is in place. Opinions of methods were varied with some participants favouring stratified blocks and others favouring minimisation. This was generally due to researchers' perception of the effect these methods had on balance and predictability. Generally, predictability was considered more important than balance as adjustments cannot be made for it; however, most researchers felt that the importance of these two methods was dependent on the design of the study. Balance is usually evaluated by tabulating variables by treatment arm and looking for perceived imbalances, predictability was generally considered much harder to measure, partly due to differing definitions.

CONCLUSION

There is a wide variety in practice on how randomisation methods are selected and researcher's opinions on methods. The difference in practice observed when looking at randomisation method selection can be explained by a difference in unit practice, and also by a difference in researchers prioritisation of balance and predictability. The findings of this study show a need for more guidance on randomisation method selection.

Taxing for a better life? The impact of environmental taxes on income distribution and inclusive education

This paper examines the impact of environmental taxes on economic and social inequalities using data from 38 OECD countries from 1994 to 2020. The results show that the introduction of an environmental tax can have unequal consequences on population groups due to differences in consumption behaviour and access to environmental alternatives. The results also indicate that environmental taxes with a progressive character (i.e. higher for higher income households) can reduce inequalities and improve environmental efficiency. The introduction of environmental taxes should therefore be done with care and with due regard to their impact on inequality. Tax policies must be designed to protect the most vulnerable households and promote equity while protecting the environment. Thus, environmental taxation should be accompanied by social and economic policies that reduce inequalities and support the most affected social groups. It is also important for governments to have better communication and awareness-raising on the impacts of environmental taxation on inequalities, in order to ensure a just transition towards sustainable lifestyles.

Recruiting women with ductal carcinoma in situ to a randomised controlled trial: lessons from the LORIS study

BACKGROUND

The LOw RISk DCIS (LORIS) study was set up to compare conventional surgical treatment with active monitoring in women with ductal carcinoma in situ (DCIS). Recruitment to trials with a surveillance arm is known to be challenging, so strategies to maximise patient recruitment, aimed at both patients and recruiting centres, were implemented.

METHODS

Women aged ≥ 46 years with a histologically confirmed diagnosis of non-high-grade DCIS were eligible for 1:1 randomisation to either surgery or active monitoring. Prior to randomisation, all eligible women were invited to complete: (1) the Clinical Trials Questionnaire (CTQ) examining reasons for or against participation, and (2) interviews exploring in depth opinions about the study information sheets and film. Women agreeing to randomisation completed validated questionnaires assessing health status, physical and mental health, and anxiety levels. Hospital site staff were invited to communication workshops and refresher site initiation visits to support recruitment. Their perspectives on LORIS recruitment were collected via surveys and interviews.

RESULTS

Eighty percent (181/227) of eligible women agreed to be randomised. Over 40% of participants had high anxiety levels at baseline. On the CTQ, the most frequent most important reasons for accepting randomisation were altruism and belief that the trial offered the best treatment, whilst worries about randomisation and the influences of others were the most frequent most important reasons for declining. Most women found the study information provided clear and useful. Communication workshops for site staff improved knowledge and confidence but only about half said they themselves would join LORIS if eligible. The most common recruitment barriers identified by staff were low numbers of eligible patients and patient preference.

CONCLUSIONS

Recruitment to LORIS was challenging despite strategies aimed at both patients and site staff. Ensuring that recruiting staff support the study could improve recruitment in similar future trials.

TRIAL REGISTRATION

ISRCTN27544579, prospectively registered on 22 May 2014.

Controlled amplification in oncology dose-finding trials

In oncology clinical trials the guiding principle of model-based dose-finding designs for cytotoxic agents is to progress as fast as possible towards, and identify, the dose level most likely to be the MTD. Recent developments with non-cytotoxic agents have broadened the scope of early phase trials to include multiple objectives. The ultimate goal of dose-finding designs in our modern era is to collect the relevant information in the study for final RP2D determination. While some information is collected on dose levels below and in the vicinity of the MTD during the escalation (using conventional tools such as the Continual Reassessment Method for example), designs that include expansion cohorts or backfill patients effectively amplify the information collected on the lower dose levels. This is achieved by allocating patients to dose levels slightly differently during the study in order to take into account the possibility that "less (dose) might be more". The objective of this paper is to study the concept of amplification. Under the heading of controlled amplification we can include dose expansion cohorts and backfill patients among others. We make some general observations by defining these concepts more precisely and study a specific design that exploits the concept of controlled amplification.

Student and the Lanarkshire milk experiment

A detailed examination of the 1930 Lanarkshire Milk Experiment (LME) by the famous statistician William Sealy Gossett ("Student"), which appeared in Biometrika in 1931, is re-examined from a more modern perspective. The LME had a complicated design whereby 67 schools in Lanarkshire were allocated to receive either raw or pasteurised milk but pupils within the schools were allocated to either receive milk or to act as controls. Student's criticisms are considered in detail and examined in terms of subsequent developments on the design and analysis of experiments, in particular as regards appropriate estimation of standard errors of treatment estimates when an incomplete blocks structure has been used. An analogy with a more modern trial in osteoarthritis is made. Suggestions are made as to how analysis might proceed if the original data were available. Some lessons for observational studies in epidemiology are drawn and it is speculated that hidden clustering structures might be an explanation as to why results may vary from observational study to observational study by more than conventionally calculated standard errors might suggest.

A systematic review of randomisation method use in RCTs and association of trial design characteristics with method selection

BACKGROUND

When conducting a randomised controlled trial, there exist many different methods to allocate participants, and a vast array of evidence-based opinions on which methods are the most effective at doing this, leading to differing use of these methods. There is also evidence that study characteristics affect the performance of these methods, but it is unknown whether the study design affects researchers' decision when choosing a method.

METHODS

We conducted a review of papers published in five journals in 2019 to assess which randomisation methods are most commonly being used, as well as identifying which aspects of study design, if any, are associated with the choice of randomisation method. Randomisation methodology use was compared with a similar review conducted in 2014.

RESULTS

The most used randomisation method in this review is block stratification used in 162/330 trials. A combination of simple, randomisation, block randomisation, stratification and minimisation make up 318/330 trials, with only a small number of more novel methods being used, although this number has increased marginally since 2014. More complex methods such as stratification and minimisation seem to be used in larger multicentre studies.

CONCLUSIONS

Within this review, most methods used can be classified using a combination of simple, block stratification and minimisation, suggesting that there is not much if any increase in the uptake of newer more novel methods. There seems to be a noticeable polarisation of method use, with an increase in the use of simple methods, but an increase in the complexity of more complex methods, with greater numbers of variables included in the analysis, and a greater number of strata.

A multinational, phase 2, randomised, adaptive protocol to evaluate immunogenicity and reactogenicity of different COVID-19 vaccines in adults ≥75 already vaccinated against SARS-CoV-2 (EU-COVAT-1-AGED): a trial conducted within the VACCELERATE network

Background

In the ongoing COVID-19 pandemic, advanced age is a risk factor for a severe clinical course of SARS-CoV-2 infection. Thus, older people may benefit in particular from booster doses with potent vaccines and research should focus on optimal vaccination schedules. In addition to each individual’s medical history, immunosenescence warrants further research in this population. This study investigates vaccine-induced immune response over 1 year.

Methods/design

EU-COVAT-1-AGED is a randomised controlled, adaptive, multicentre phase II protocol evaluating different booster strategies in individuals aged ≥75 years (n=600) already vaccinated against SARS-CoV-2. The initial protocol foresaw a 3rd vaccination (1st booster) as study intervention. The present modified Part B of this trial foresees testing of mRNA-1273 (Spikevax®) vs. BNT162b2 (Comirnaty®) as 4th vaccination dose (2nd booster) for comparative assessment of their immunogenicity and safety against SARS-CoV-2 wild-type and variants. The primary endpoint of the trial is to assess the rate of 2-fold antibody titre increase 14 days after vaccination measured by quantitative enzyme-linked immunosorbent assay (Anti-RBD-ELISA) against wild-type virus. Secondary endpoints include the changes in neutralising antibody titres (Virus Neutralisation Assay) against wild-type as well as against Variants of Concern (VOC) at 14 days and up to 12 months. T cell response measured by qPCR will be performed in subgroups at 14 days as exploratory endpoint. Biobanking samples are being collected for neutralising antibody titres against potential future VOC. Furthermore, potential correlates between humoral immune response, T cell response and neutralising capacity will be assessed.

The primary endpoint analysis will be triggered as soon as for all patients the primary endpoint (14 days after the 4th vaccination dose) has been observed.

Discussion

The EU-COVAT-1-AGED trial Part B compares immunogenicity and safety of mRNA-1273 (Spikevax®) and BNT162b2 (Comirnaty®) as 4th SARS-CoV-2 vaccine dose in adults ≥75 years of age. The findings of this trial have the potential to optimise the COVID-19 vaccination strategy for this at-risk population.

Trial registration

ClinicalTrials.govNCT05160766. Registered on 16 December 2021.

Protocol version: V06_0: 27 July 2022

Supplementary Information

The online version contains supplementary material available at 10.1186/s13063-022-06791-y.

A pilot single-blind parallel randomised controlled trial comparing kinesiology tape to compression in the management of subacute hand oedema after trauma

Background

Hand oedema is a common consequence of hand trauma or surgery. There are numerous methods to reduce hand oedema but lack high-quality evidence to support best practice. The primary objective of this pilot trial was to assess study feasibility when comparing treatments for subacute hand oedema after trauma.

Methods

A parallel two-arm pilot randomised controlled trial was conducted in the hand therapy department at a regional hospital in Norfolk between October 2017 and July 2018. Patients were eligible if 18 years or over, referred to hand therapy with subacute hand oedema. Randomisation was on a 1:1 basis to treatment as usual (TAU) (compression, elevation and massage) or trial treatment (TT) (kinesiology tape, elevation and massage). One blinded assessor completed all assessments (prior to randomisation, 4 and 12 weeks later). Data on study feasibility, adherence and acceptability of treatments were collected. The primary outcome measure was hand volume (volumetry). Patient-rated severity (0–5 Likert scale), hand health profile of the Patient Evaluation Measure (PEM) and quality of life (EQ-5D-5L) were also recorded.

Results

Forty-five patients were screened for eligibility and 26 consented and were randomised with 13 patients in each treatment arm. Twelve participants were lost to follow-up leaving 7 participants in each group included in the analysis. Assessor blinding was maintained in 64% of participants (9/14). Total mean acceptability scores, out of 100, were higher for TAU (87.9) than TT (76.1). Health resource use results showed TT was marginally cheaper (~£2 per patient) than TAU.

Individual adherence ranged between 39 and 100%, with higher levels of overall adherence seen in the TAU group. Four participants (28%) reported adverse effects (TT group n = 3, TAU group n = 1).

Conclusion

This pilot trial has identified that modifications are required in order to make a full-scale trial feasible. They include a formal assessment of treatment fidelity, research staff assisting with screening and recruitment of participants and multiple blinded assessors at each study site. Whilst not designed as an efficacy trial, it should be acknowledged that the small sample size and high loss to follow-up meant very small numbers were included in the final analysis resulting in wide confidence intervals and therefore low precision in parameter estimates.

Trial registration

International Standard Randomised Controlled Trial Number: 94083271. Date of registration 16th August 2017.

Trial funding

National Institute for Health Research Trainees Co-ordinating Centre (TCC); Grant Codes: CDRF-2014-05-064

Supplementary Information

The online version contains supplementary material available at 10.1186/s40814-022-01023-1.

A randomised controlled multicentre investigator-blinded clinical trial comparing efficacy and safety of surgery versus complex physical decongestive therapy for lipedema (LIPLEG)

BACKGROUND

Lipedema is a chronic disorder of the adipose tissue that affects mainly women, characterised by symmetrical, excessive fatty tissue on the legs and pain. Standard conservative treatment is long-term comprehensive decongestive therapy (CDT) to alleviate lipedema-related pain and to improve psychosocial well-being, mobility and physical activity. Patients may benefit from surgical removal of abnormally propagated adipose tissue by liposuction. The LIPLEG trial evaluates the efficacy and safety of liposuction compared to standard CDT.

METHODS/DESIGN

LIPLEG is a randomised controlled multicentre investigator-blinded trial. Women with lipedema (n=405) without previous liposuction will be allocated 2:1 to liposuction or CDT. The primary outcome of the trial is leg pain reduction by ≥2 points on a visual analogue scale ranging 0-10 at 12 months on CDT or post-completion of liposuction. Secondary outcomes include changes in leg pain severity, health-related quality of life, depression tendency, haematoma tendency, prevalence of oedema, modification physical therapy scope, body fat percentage, leg circumference and movement restriction. The primary analysis bases on intention-to-treat. Success proportions are compared using the Mantel-Haenszel test stratified by lipedema stage at a 5% two-sided significance level. If this test is statistically significant, the equality of the response proportions in the separate strata is evaluated by Fisher's exact test in a hierarchical test strategy.

DISCUSSION

LIPLEG assesses whether surgical treatment of lipedema is safe and effective to reduce pain and other lipedema-related health issues. The findings of this trial have the potential to change the standard of care in lipedema.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04272827. Registered on February 14, 2020.

TRIAL STATUS

Protocol version is 02_0, December 17, 2019.

Does the number of doses matter? A qualitative study of HPV vaccination acceptability nested in a dose reduction trial in Tanzania

Background

The multi-dose regimen is a known barrier to successful human papillomavirus (HPV) vaccination. Emerging evidence suggests that one vaccine dose could protect against HPV. While there are clear advantages to a single dose schedule, beliefs about vaccine dosage in low and middle income countries (LMICs) are poorly understood. We investigated acceptability of dose-reduction among girls, and parents/guardians of girls, randomised to receive one, two or three doses in an HPV vaccine dose-reduction and immunobridging study (DoRIS trial) in Tanzania.

Methods

Semi-structured interviews with girls (n = 19), and parents/guardians of girls (n = 18), enrolled in the study and completing their vaccine course.

Results

Most participants said they entrusted decisions about the number of HPV vaccine doses to experts. Random allocation to the different dose groups did not feature highly in the decision to participate in the trial. Given a hypothetical choice, girls generally said they would prefer fewer doses in order to avoid the pain of injections. Parental views were mixed, with most wanting whichever dose was most efficacious. Nonetheless, a few parents equated a higher number of doses with greater protection.

Conclusion

Vaccine trials and programmes will need to employ careful messaging to explain that one dose offers sufficient protection against HPV should emerging evidence from ongoing dose-reduction clinical trials support this.

•

We interviewed girls, and parents/carers of girls, enrolled in an HPV Vaccine dose reduction trial.

•

We found that enrolling in the trial in the context of community rumours required trust in the trial scientists.

•

Scientists were trusted to decide on dosage; thus randomisation by dosage was not an acceptability issue.

•

Girls preferred fewer vaccine doses in order to avoid injection-related pain.

•

Parents/guardians generally wanted whichever dose regimen was most efficacious.

[A short history of the concept of bias in epidemiology]

BACKGROUND

Bias is a major methodological issue for epidemiology. However, only a few studies have been dedicated to the past and present formulations of the concept of bias. Moreover, the classical definition of bias as systematic deviation from the truth of results or inferences, definition which can be found in dictionaries of epidemiology, does not seem to either match the way epidemiologists use it in practice, or correspond to the different definitions given throughout its history. It is consequently important to elucidate this paradox.

METHODS

In this historical and conceptual article, we study the different uses of the word "bias" in epidemiological literature, from classic articles in the 1950's about the link between smoking and lung cancer to the most recent epidemiology textbooks, the objective being to analyze the ways in which epidemiologists have defined, applied and modified this concept over time.

RESULTS

We show that D.L. Sackett's article on bias in analytic research, published in 1979, put an end, at least temporarily, to reflection in populational epidemiology that started thirty years before. More precisely, we show that Sackett's definition of bias corresponds more to the needs and goals of clinical epidemiology than to those of populational epidemiology. Concomitantly, populational epidemiologists such as K.J. Rothman redefined bias as a threat to the internal validity of a study, and epidemiological study as an "exercise in measurement of an effect rather than as a criterion-guided process for deciding whether an effect is present or not".

CONCLUSION

It is thereby important to draw a distinction between two notions pertaining to bias: an epidemiological concept of bias, viewed as the lack of internal validity of an observational study; and a medical concept of bias, defined as deviation from the truth. The former concerns the design and methodology of epidemiological studies; the latter is more general and impels epidemiologists and physicians to be skeptical, and even critical, towards their own inferences.

A systematic review of antimicrobial susceptibility testing as a tool in clinical trials assessing antimicrobials against infections due to gram-negative pathogens

BACKGROUND

Antimicrobial susceptibility testing (AST) is the standard of care for treating bacterial infections. In randomized clinical trials of new antimicrobials, AST might not be performed or reported in real time.

OBJECTIVES

To determine local, real-time laboratory AST performance, its usage in the trial flow, quality control (QC) of the local testing, central AST performance and the effect of using AST categorization on the trials' primary outcomes.

DATA SOURCES

We systematically searched PubMed, Embase, PsychINFO and Web of Science.

ELIGIBILITY CRITERIA

We included registered randomized controlled trials published in journals between January 2015 and December 2019.

PARTICIPANTS AND INTERVENTIONS

We included trials comparing different antibiotics for the treatment of infections caused predominantly by Gram-negative bacteria.

METHODS

Primary outcomes for different trial populations were extracted and differences between trial arms were compared for patients with infections caused by susceptible versus non-susceptible bacteria. Results are described narratively.

RESULTS

Of 32 randomized trials, 25 trials reported that local AST was performed, 1312 reported the local laboratory AST methods, no trial reported QC, but post-hoc referral for AST at a reference laboratory was common. Patients' outcomes were superior when patients with infections due to susceptible and non-susceptible pathogens were compared post hoc (median difference 14%, interquartile range 8%-24%) in trials allowing this comparison (seven antimicrobials), except for colistin, where 14-day mortality was 9% higher when patients were treated with colistin for colistin-susceptible versus colistin-resistant carbapenem-resistant Acinetobacter baumannii. When excluding patients with pathogens that were non-susceptible to either antimicrobial in the trials, the difference in the primary outcome between the trial arms was reduced in five out of six trials.

CONCLUSIONS

Trials should perform AST to guide patient inclusion or exclusion from the study and consider the impact of the central laboratory susceptibility results on the study outcomes when using post-hoc reference testing.

Preserving equipoise and performing randomised trials for COVID-19 social distancing interventions

In the coronavirus disease 2019 (COVID-19) pandemic, a large number of non-pharmaceutical measures that pertain to the wider group of social distancing interventions (e.g. public gathering bans, closures of schools, workplaces and all but essential business, mandatory stay-at-home policies, travel restrictions, border closures and others) have been deployed. Their urgent deployment was defended with modelling and observational data of spurious credibility. There is major debate on whether these measures are effective and there is also uncertainty about the magnitude of the harms that these measures might induce. Given that there is equipoise for how, when and if specific social distancing interventions for COVID-19 should be applied and removed/modified during reopening, we argue that informative randomised-controlled trials are needed. Only a few such randomised trials have already been conducted, but the ones done to-date demonstrate that a randomised trials agenda is feasible. We discuss here issues of study design choice, selection of comparators (intervention and controls), choice of outcomes and additional considerations for the conduct of such trials. We also discuss and refute common counter-arguments against the conduct of such trials.

Interpretation of within-group change in randomised trials

In medicine, it is common to observe improvement after intervention, at least partly because patients present for care in extremis and would have improved without intervention. Controlling for this counterfactual explanation for improvement is the principle reason to conduct a trial in which patients are randomised to treatment or a control group. Accordingly, it is not reasonable to infer that both interventions are effective when the groups show similar improvements in outcome.

Dynamic LED light versus static LED light for depressed inpatients: results from a randomized feasibility trial

Background

Retrospective studies conducted in psychiatric wards have indicated a shorter duration of stay for depressed inpatients in bright compared to dim daylight-exposed rooms, pointing to a possible antidepressant effect of daylight conditions. Dynamic LED lighting, aiming to mimic daylight conditions, are currently been installed in several hospitals, but their feasibility is poorly investigated.

Methods

To investigate the feasibility of these systems, we developed and installed a LED-lighting system in four rooms in a psychiatric inpatient ward. The system could function statically or dynamically regarding light intensity and colour temperature. The system consisted of (A) a large LED luminaire built into the window jamb mimicking sunlight reflections, (B) two LED light luminaires in the ceiling and (C) a LED reading luminaire. In the static mode, the systems provided constant light from A and B. In the dynamic mode, the system changed light intensity and colour temperature using A, B and C. Patients with unipolar or bipolar depression were randomised to dynamic or static LED lighting for 4 weeks, in addition to standard treatment. Primary outcome was the rate of patients discontinuing the trial due to discomfort from the lighting condition. Secondary outcomes were recruitment and dropout rates, visual comfort, depressive symptoms and suicidal ideation.

Results

No participants discontinued due to discomfort from the LED lighting. Recruitment rate was 39.8%, dropout from treatment rates were 56.3% in the dynamic group and 33.3% in the static group. 78.1% in the dynamic group were satisfied with the lighting compared with 71.8% in the static group. Discomfort from the light (glare) was reported by 11.5% in the dynamic group compared to 5.1% in the static group. Endpoint suicidal scores were 16.8 (10.4) in the dynamic and 16.3 (14.9) in the static group. The lighting system was 100% functional. The light sensor system proved unstable.

Conclusion

Dropout from treatment was high primarily due to early discharge and with a lack of endpoint assessments. The feasibility study has influenced an upcoming large-scale dynamic lighting efficacy trial where we will use a shorter study period of 3 weeks and with more emphasis on endpoint assessments. The lighting was well tolerated in both groups, but some found intensity too low in the evening. Thus, we will use higher intensity blue-enriched light in the morning and higher intensity amber (blue-depleted) light in the evening in the upcoming study. The light sensor system needs to be improved

Trial registration

ClinicalTrials.gov: NCT03363529

Randomised Controlled Trials for Informing Perinatal Care

BACKGROUND

Randomised controlled trials provide the best evidence for the effects of interventions and are a key tool in the effort to improve the care and outcomes for newborn infants.

METHODS

We discuss the role of randomisation for minimising selection bias in clinical trials and describe examples of seminal trials that have shaped the development of modern perinatal care. We consider the challenges inherent in designing and delivering large, simple, and pragmatic trials, and the need for the development and adoption of core outcome sets to ensure that trials provide high-quality evidence of sufficient validity and applicability to guide policy and practice.

RESULTS

Since the earliest days of modern neonatology, the randomised controlled trial has been recognised as the best method for assessing treatments and practices. While many strategies that reduce mortality and morbidity have been introduced following randomised trials, there are, however, important examples of ineffective or potentially harmful practices that have been adopted in the absence of trial-based evidence. Typically, randomised controlled trials in perinatal care need to recruit several thousand participants to be able to detect modest but potentially important effects of new interventions on the most important but rare outcomes. Given the concerns about the financial burden and regulatory complexity of standard trial designs, innovative "efficient" trial designs are being evaluated to streamline processes while safeguarding participants.

CONCLUSIONS

Well-conducted randomised controlled trials provide the most robust evaluation of interventions aimed at improving outcomes for newborn infants and their families. Increasingly, these trials will need to be large and multicentre (often international) and use a simple and pragmatic protocol, incorporating meticulous follow-up procedures and assessment of long-term outcomes.

Community perspectives on randomisation and fairness in a cluster randomised controlled trial in Zambia

BACKGROUND

One important ethical issue in randomised controlled trials (RCTs) is randomisation. Relatively little is known about how participating individuals and communities understand and perceive central aspects of randomisation such as equality, fairness, transparency and accountability in community-based trials. The aim of this study was to understand and explore study communities' perspectives of the randomisation process in a cluster RCT in rural Zambia studying the effectiveness of different support packages for adolescent girls on early childbearing.

METHODS

In this explorative study, in-depth semi-structured interviews were carried out in 2018 with 14 individuals who took part in the randomisation process of the Research Initiative to Support the Empowerment of Girls (RISE) project in 2016 and two traditional leaders. Two of the districts where the trial is implemented were purposively selected. Interviews were audio recorded and fully transcribed. Data were analysed by coding and describing emergent themes.

RESULTS

The understanding of the randomisation process varied. Some respondents understood that randomisation was conducted for research purposes, but most of them did not. They had trouble distinguishing research and aid. Generally, respondents perceived the randomisation process as transparent and fair. However, people thought that there should not have been a "lottery" because they wanted all schools to receive equal or balanced benefits of the interventions.

CONCLUSIONS

Randomisation was misunderstood by most respondents. Perceived procedural fairness was easier to realize than substantive fairness. Researchers working on Cluster Randomised Controlled Trials (CRCTs) should consider carefully how to explain randomisation.

An introduction to clinical trial design

Clinicians and other decision makers in healthcare use results from clinical trials to inform practice. Interpretation of clinical trial results can be challenging, as weaknesses in trial design, data collection, analysis or reporting, can compromise the usefulness of results. A good working knowledge of clinical trial design is essential to expertly interpret and determine the validity and generalizability of the results. This manuscript will give a brief overview of clinical trial design including the strengths and limitations of various approaches. The focus will be on confirmatory clinical trials.

Patient experiences of participation in a radical thoracic surgical trial: findings from the Mesothelioma and Radical Surgery Trial 2 (MARS 2)

BACKGROUND

The Mesothelioma and Radical Surgery Trial (MARS 2) aims to evaluate a surgical procedure by comparing chemotherapy and surgery against chemotherapy alone. The pilot study for MARS 2 evaluated the viability of recruitment. Challenges have been reported in conducting clinical research into thoracic surgical treatments and evidence is required to improve our understanding of patient experiences of trial procedures, trial treatments and the factors that influence participation.

METHODS

This longitudinal qualitative study was nested within the MARS 2 pilot. Semi-structured telephone interviews were conducted with 15 participants in the MARS 2 trial. Interviews were conducted post-randomisation, post-surgery (surgery arm) and at 6 and 12 months. Altogether, 41 interviews were carried out. The data were analysed using framework techniques.

RESULTS

Challenges were identified regarding the volume and complexity of information given to participants, and their understanding of clinical equipoise and randomisation. Factors influencing participation included having an opportunity to undergo surgery, a self-assessment of their ability to cope with trial treatments, maintaining a positive approach and altruism. Obstacles included the logistics of traveling for treatment in an unfamiliar setting. Negative consequences of trial participation included increased uncertainty amplified by multiple care providers and unclear transition arrangements after the trial.

CONCLUSIONS

Participants' descriptions provided insights that have implications for care for mesothelioma trial patients. The need for healthcare staff to be alert to the potential for misunderstanding, particularly when presenting treatment options, was identified. Patients perceived and derived benefits from taking part in the trial but experienced some negative consequences. These should be anticipated and managed proactively.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT02040272 . Registered on 20 January 2014.

Hope and despair: a qualitative exploration of the experiences and impact of trial processes in a rehabilitation trial

BACKGROUND

Unanticipated responses by research participants can influence randomised controlled trials (RCTs) in multiple ways, many of which are poorly understood. This study used qualitative interviews as part of an embedded process evaluation to explore the impact participants may have on the study, but also unintended impacts the study may have on them.

AIM

The aim of the study was to explore participants' experiences and the impact of trial involvement in a pilot RCT in order to inform the designing and delivery of a definitive RCT.

METHODS

In-depth interviews with 20 participants (10 in the intervention and 10 in the control group) enrolled in a stroke rehabilitation pilot trial. A modified framework approach was used to analyse transcripts.

RESULTS

Participation in the study was motivated partly by a desperation to receive further rehabilitation after discharge. Responses to allocation to the control group included an increased commitment to self-treatment, and negative psychological consequences were also described. Accounts of participants in both control and intervention groups challenge the presumption that they were neutral, or in equipoise, regarding group allocation prior to consenting to randomisation.

CONCLUSIONS

Considering and exploring participant and participation effects, particularly in the control group, highlights numerous issues in the interpretation of trial studies, as well as the in ethics of RCTs more generally. While suggestions for a definitive trial design are given, further research is required to investigate the significant implications these findings may have for trial design, monitoring and funding.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT02429180 . Registered on 29 April/2015.

Risk of bias judgments for random sequence generation in Cochrane systematic reviews were frequently not in line with Cochrane Handbook

BACKGROUND

Assessing the risk of bias (RoB) in included studies is one of the key methodological aspects of systematic reviews. Cochrane systematic reviews appraise RoB of randomised controlled trials (RCTs) with the Cochrane RoB tool. Detailed instructions for using the Cochrane RoB tool are provided in the Cochrane Handbook for Systematic Reviews of Interventions (The Cochrane Handbook). The purpose of this study was to analyse whether Cochrane authors use adequate judgments about the RoB for random sequence generation of RCTs included in Cochrane reviews.

METHODS

We extracted authors' judgments (high, low or unclear RoB) and supports for judgments (comments accompanying judgments which explain the rationale for a judgment) for random sequence generation of included RCTs from RoB tables of Cochrane reviews using automated data scraping. We categorised all supporting comments, analysed the number and type of various supporting comments and assessed adequacy of RoB judgment for randomisation in line with recommendations from the Cochrane Handbook.

RESULTS

We analysed 10,103 RCTs that were included in 704 Cochrane reviews. For 5,706 RCTs, randomisation was not described, but for the remaining RCTs, it was indicated that randomisation was performed using computer/software/internet (N = 2,850), random number table (N = 883), mechanical method (N = 359) or it was incomplete/inappropriate (N = 305). Overall, 1,220/10,103 trials (12%) did not have a RoB judgment in line with Cochrane Handbook guidance about randomisation. The highest proportion of misjudgements was found for trials with high RoB (28%), followed by those with low (20%) or unclear (3%). Therefore, one in eight judgments for the analysed domain in Cochrane reviews was not in line with Cochrane Handbook, and one in four if the judgment was "high risk".

CONCLUSION

Authors of Cochrane reviews often make judgments about the RoB related to random sequence generation that are not in line with instructions given in the Cochrane Handbook, which compromises the reliability of the systematic reviews. Our results can help authors of both Cochrane and non-Cochrane reviews which use Cochrane RoB tool to avoid making common mistakes when assessing RoB in included trials.

Changing platforms without stopping the train: experiences of data management and data management systems when adapting platform protocols by adding and closing comparisons

Background

There is limited research and literature on the data management challenges encountered in multi-arm, multi-stage platform and umbrella protocols. These trial designs allow both (1) seamless addition of new research comparisons and (2) early stopping of accrual to individual comparisons that do not show sufficient activity. FOCUS4 (colorectal cancer) and STAMPEDE (prostate cancer), run from the Medical Research Council Clinical Trials Unit (CTU) at UCL, are two leading UK examples of clinical trials implementing adaptive platform protocol designs. To date, STAMPEDE has added five new research comparisons, closed two research comparisons following pre-planned interim analysis (lack of benefit), adapted the control arm following results from STAMPEDE and other relevant trials, and completed recruitment to six research comparisons. FOCUS4 has closed one research comparison following pre-planned interim analysis (lack of benefit) and added one new research comparison, with a number of further comparisons in the pipeline. We share our experiences from the operational aspects of running these adaptive trials, focusing on data management.

Methods

We held discussion groups with STAMPEDE and FOCUS4 CTU data management staff to identify data management challenges specific to adaptive platform protocols. We collated data on a number of case report form (CRF) changes, database amendments and database growth since each trial began.

Discussion

We found similar adaptive protocol-specific challenges in both trials. Adding comparisons to and removing them from open trials provides extra layers of complexity to CRF and database development. At the start of an adaptive trial, CRFs and databases must be designed to be flexible and scalable in order to cope with the continuous changes, ensuring future data requirements are considered where possible. When adding or stopping a comparison, the challenge is to incorporate new data requirements while ensuring data collection within ongoing comparisons is unaffected. Some changes may apply to all comparisons; others may be comparison-specific or applicable only to patients recruited during a specific time period. We discuss the advantages and disadvantages of the different approaches to CRF and database design we implemented in these trials, particularly in relation to use and maintenance of generic versus comparison-specific CRFs and databases. The work required to add or remove a comparison, including the development and testing of changes, updating of documentation, and training of sites, must be undertaken alongside data management of ongoing comparisons. Adequate resource is required for these competing data management tasks, especially in trials with long follow-up. A plan is needed for regular and pre-analysis data cleaning for multiple comparisons that could recruit at different rates and periods of time. Data-cleaning activities may need to be split and prioritised, especially if analyses for different comparisons overlap in time.

Conclusions

Adaptive trials offer an efficient model to run randomised controlled trials, but setting up and conducting the data management activities in these trials can be operationally challenging. Trialists and funders must plan for scalability in data collection and the resource required to cope with additional competing data management tasks.

Electronic supplementary material

The online version of this article (10.1186/s13063-019-3322-7) contains supplementary material, which is available to authorized users.

Heterogeneity of treatment effect by baseline risk of mortality in critically ill patients: re-analysis of three recent sepsis and ARDS randomised controlled trials

Background

Randomised controlled trials (RCTs) enrolling patients with sepsis or acute respiratory distress syndrome (ARDS) generate heterogeneous trial populations. Non-random variation in the treatment effect of an intervention due to differences in the baseline risk of death between patients in a population represents one form of heterogeneity of treatment effect (HTE). We assessed whether HTE in two sepsis and one ARDS RCTs could explain indeterminate trial results and inform future trial design.

Methods

We assessed HTE for vasopressin, hydrocortisone and levosimendan in sepsis and simvastatin in ARDS patients, on 28-day mortality, using the total Acute Physiology And Chronic Health Evaluation II (APACHE II) score as the baseline risk measurement, comparing above (high) and below (low) the median score. Secondary risk measures were the acute physiology component of APACHE II and predicted risk of mortality using the APACHE II score. HTE was quantified both in additive (difference in risk difference (RD)) and multiplicative (ratio of relative risks (RR)) scales using estimated treatment differences from a logistic regression model with treatment risk as the interaction term.

Results

The ratio of the odds of death in the highest APACHE II quartile was 4.9 to 7.4 times compared to the lowest quartile, across the three trials. We did not observe HTE for vasopressin, hydrocortisone and levosimendan in the two sepsis trials. In the HARP-2 trial, simvastatin reduced mortality in the low APACHE II group and increased mortality in the high APACHE II group (difference in RD = 0.34 (0.12, 0.55) (p = 0.02); ratio of RR 3.57 (1.77, 7.17) (p < 0.001). The HTE patterns were inconsistent across the secondary risk measures. The sensitivity analyses of HTE effects for vasopressin, hydrocortisone and levosimendan were consistent with the main analyses and attenuated for simvastatin.

Conclusions

We assessed HTE in three recent ICU RCTs, using multivariable baseline risk of death models. There was considerable within-trial variation in the baseline risk of death. We observed potential HTE for simvastatin in ARDS, but no evidence of HTE for vasopressin, hydrocortisone or levosimendan in the two sepsis trials. Our findings could be explained either by true lack of HTE (no benefit of vasopressin, hydrocortisone or levosimendan vs comparator for any patient subgroups) or by lack of power to detect HTE. Our results require validation using similar trial databases.

Electronic supplementary material

The online version of this article (10.1186/s13054-019-2446-1) contains supplementary material, which is available to authorized users.

How to maintain the maximal level of blinding in randomisation for a placebo-controlled drug trial

We illustrate the approach of randomising treatments and compare it with the traditional approach of randomising patients, using a case study drawn from the authors’ experience in clinical trials. The setting is a double-blind parallel two-arm randomised controlled trial (RCT), but the method in this paper can be extended to single-blind, cross-over, or multi-arm RCTs.

We propose the concept of two different levels of blinding: full blinding and partial blinding. We subsequently show how to maintain the maximal level of blinding. Using an example, we show that a pharmacist can be fully blinded if the investigational medical products (IMPs) that they prescribe (instead of patients) are randomised, and they can be partially blinded if they need to dispense replacement (i.e., surplus) IMPs. A small number of surplus IMPs is commonly required in a clinical trial to replace lost or damaged IMPs. We note that the concept of full blinding and partial blinding is different from double-blind trial, and the level of blinding is relevant in both single-blind and double-blind trials.

A trial statistician needs to work closely with all parties in the design of the randomisation, including the pharmacist, the trial manager, and the manufacturer. We detail what should and should not be shown in the various documents that the trial statistician need to provide to the pharmacist and to the manufacturer. We provide template tables for these documents.

Risk of bias assessment of sequence generation: a study of 100 systematic reviews of trials

BACKGROUND

Systematic reviews of randomised trials guide policy and healthcare decisions. Yet, we observed that some reviews judge randomised trials as high or unclear risk of bias (ROB) for sequence generation, potentially introducing bias. However, to date, the extent of this issue has not been well examined. We evaluated the consistency in the ROB assessment for sequence generation of randomised trials in Cochrane and non-Cochrane reviews, and explored the reviewers' judgement of the quality of evidence for the related outcomes.

METHODS

Cochrane intervention reviews (01/01/2017-31/03/2017) were retrieved from the Cochrane Database of Systematic Reviews. We also searched for systematic reviews in ten general medical journals with highest impact factors (01/01/2016-31/03/2017). We examined the proportion of reviews that rated the sequence generation domain as high, low or unclear risk of selection bias. For reviews that had rated any randomised trials as high or unclear risk of bias, we examined the proportion that had assessed the quality of evidence.

RESULTS

Overall, 100 systematic reviews were included in our analysis. We evaluated 64 Cochrane reviews which comprised of 984 randomised trials; 0.8% (n = 8) and 52.2% (n = 514) were rated as high and unclear ROB for sequence generation respectively. We further evaluated 36 non-Cochrane reviews which comprised of 1376 trials; 5.8% (n = 80) and 39.6% (n = 545) were rated as high and unclear ROB respectively. Ninety percent (n = 10) of non-Cochrane reviews which rated randomised trials as high ROB for sequence generation did not report an underlying reason. All Cochrane reviews assessed the quality of evidence (GRADE). For the non-Cochrane reviews, only just over half had assessed the quality of evidence.

CONCLUSION

Systematic reviews of interventions frequently rate randomised trials as high or unclear ROB for sequence generation. In general, Cochrane reviews were more transparent than non-Cochrane reviews in ROB and quality of evidence assessment. The scientific community should more strongly promote consistent ROB assessment for sequence generation to minimise selection bias and support transparent quality of evidence assessment. Consistency ensures that appropriate conclusions are drawn from the data.

The Shiny Balancer - software and imbalance criteria for optimally balanced treatment allocation in small RCTs and cRCTs

Background

In randomised controlled trials with only few randomisation units, treatment allocation may be challenging if balanced distributions of many covariates or baseline outcome measures are desired across all treatment groups. Both traditional approaches, stratified randomisation and allocation by minimisation, have their own limitations. A third method for achieving balance consists of randomly choosing from a preselected list of sufficiently balanced allocations. As with minimisation, this method requires that heterogeneity between treatment groups is measured by specified imbalance metrics. Although certain imbalance measures are more commonly used than others, to the author's knowledge there is no generally accepted “gold standard”, neither for categorical and even less so for continuous variables.

Methods

An intuitive and easily accessible web-based software tool was developed which allows for balancing multiple variables of different types and using various imbalance metrics. Different metrics were compared in a simulation study.

Results

Using simulated data, it could be shown that for categorical variables, χ²-based imbalance measures seem to be viable alternatives to the established “quadratic imbalance” metric. For continuous variables, using the area between the empirical cumulative distribution functions or the largest difference in the three pairs of quartiles is recommended to measure imbalance. Another imbalance metric suggested in the literature for continuous variables, the (symmetrised) Kullback-Leibler divergence, should be used with caution.

Conclusion

The Shiny Balancer offers the possibility to visually explore the balancing properties of several well established or newly suggested imbalance metrics, and its use is particularly advocated in clinical studies with few randomisation units, as it is typically the case in cluster randomised trials.

Electronic supplementary material

The online version of this article (10.1186/s12874-018-0551-5) contains supplementary material, which is available to authorized users.

Prevalence of context effects: testing with a straightforward question of yesterday happiness

PURPOSE

Although contexts effects were found in responding to general happiness, it is little known how prevalent the effects can be in other measures of subjective wellbeing. If context effects are still found for a measure of subjective wellbeing that exhibits a host of features minimizing the effects, one can conclude that the effects can be quite prevalent. We aimed to assess this possibility.

METHODS

We analyzed the Indonesian Family Life Survey by exploiting the random assignment of four versions of a list of 12 yesterday affects. The random assignment established causality on firmer ground. We applied ordered probit models by sex (10,162 men and 11,531 women): the dependent variable of interest was a measure of happiness, and the independent variables of interest were four affects that immediately preceded happiness.

RESULTS

We found that when sadness immediately preceded happiness, men were 2.4% points more likely to say not at all happy and 2.5% points less likely to say very happy. The corresponding figures for women were 1.5 and 1.8% points. We, however, found no discernible context effects when boredom and anger immediately preceded happiness.

CONCLUSIONS

Context effects were still found for a measure of subjective wellbeing even when the effects were thought to be minimal. That said, the different influence of sadness versus boredom and anger suggests that there are ways to alleviate context effects.

Sequence balance minimisation: minimising with unequal treatment allocations

BACKGROUND

Minimisation ensures excellent balance between groups for several prognostic factors, even in small samples. However, its use with unequal allocation ratios has been problematic. This paper describes a new minimisation scheme named sequence balance minimisation for unequal treatment allocations.

METHODS

Treatment- and factor-balancing properties were assessed in simulation studies for two- and three-arm trials with 1:2 and 1:2:3 allocation ratios. Sample sizes were set 30, 60 and 120. The number of prognostic factors on which to achieve balance was ranged from zero (treatment totals only) to ten with two levels occurring in equal probabilities. Random elements were set at 0.95, 0.9, 0.85, 0.80, 0.7, 0.6 and 0.5. Characteristics of the randomisation distributions and the impact of changing the block size while maintaining the allocation ratio were also examined.

RESULTS

Sequence balance minimisation has good treatment- and factor-balancing capabilities, and the randomisation distribution was centred at zero for all scenarios. The mean and median number of allocations achieved were the same as the number expected in most scenarios, and including additional factors (up to ten) in the minimisation scheme had little impact on treatment balance. Treatment balance tended to depart from the target as the random element was lowered. The variability in allocations achieved increased slightly as the number of factors increased, as the random element was decreased and as the sample size increased. The mean and median factor imbalance remained tightly around zero even when the chosen factor was not included in the minimisation scheme, though the variability was greater. The variability in factor imbalance increased slightly as the random element decreased, as well as when the number of prognostic factors and sample size increased. Increasing block size while maintaining the allocation ratio improved treatment balance notably with little impact on factor imbalance.

CONCLUSIONS

Sequence balance minimisation has good treatment- and factor-balancing properties and is particularly useful for small trials seeking to achieve balance across several prognostic factors.

Subversion of allocation concealment in a randomised controlled trial: a historical case study

BACKGROUND

If the randomisation process within a trial is subverted, this can lead to selection bias that may invalidate the trial's result. To avoid this problem, it is recommended that some form of concealment should be put into place. Despite ongoing anecdotal concerns about their susceptibility to subversion, a surprising number of trials (over 10%) still use sealed opaque envelopes as the randomisation method of choice. This is likely due in part to the paucity of empirical data quantifying the potential effects of subversion. In this study we report a historical before and after study that compares the use of the sealed envelope method with a more secure centralised telephone allocation approach in order to provide such empirical evidence of the effects of subversion.

METHODS

This was an opportunistic before and after study set within a multi-centre surgical trial, which involved 654 patients from 28 clinicians from 23 centres in the UK and Ireland. Two methods of randomly allocating subjects to alternative treatments were adopted: (a) a sealed envelope system administered locally, and (b) a centralised telephone system administered by the trial co-ordination centre. Key prognostic variables were compared between randomisation methods: (a) age at trial entry, a key prognostic factor in the study, and (b) the order in which 'randomisation envelopes' were matched to subjects.

RESULTS

The median age of patients allocated to the experimental group with the sealed envelope system, was significantly lower both overall (59 vs 63 years, p < 0.01) and in particular for three clinicians (57 vs 72, p < 0.01; 33 vs 69, p < 0.001; 47 vs 72, p = 0.03). No differences in median age were found between the allocation groups for the centralised system.

CONCLUSIONS

Due to inadequate allocation concealment with the sealed envelope system, the randomisation process was corrupted for patients recruited from three clinicians. Centralised randomisation ensures that treatment allocation is not only secure but seen to be secure. Where this proves to be impossible, allocation should at least be performed by an independent third party. Unless it is an absolute requirement, the use of sealed envelopes should be discontinued forthwith.

"I didn't really understand it, I just thought it'd help": exploring the motivations, understandings and experiences of patients with advanced lung cancer participating in a non-placebo clinical IMP trial

BACKGROUND

Few studies have explored in depth the experiences of patients with advanced cancer who are participating in clinical investigational medicinal product trials. However, integrated qualitative studies in such trials are needed to enable a broader evaluation of patient experiences in the trial, with important ethical and practical implications for the design and conduct of similar trials and treatment regimes in the future.

METHODS

Ten participants were recruited from the control and intervention arms of FRAGMATIC: a non-placebo trial for patients with advanced lung cancer. Participants were interviewed at up to three time points during their time in the trial. Interviews were analysed using Interpretive Phenomenological Analysis.

RESULTS

Patients were motivated to join the trial out of hope of medical benefit and altruism. Understanding of randomisation was mixed and in some cases poor, as was appreciation of trial purpose and equipoise. The trial was acceptable to and evaluated positively by most participants; participants receiving the intervention focused on the potential treatment benefits they hoped they would receive, whilst participants in the control arm found alternative reasons, such as altruism, personal fulfilment and positive attention, to commit to and perceive benefits from the trial. However, whilst experiences were generally very positive, poor understanding, limited engagement with trial information and focus on treatment benefits amongst some participants give cause for concern.

CONCLUSIONS

By exploring longitudinally the psychological, emotional and cognitive domains of trial participation, we consider potential harms and benefits of participation in non-placebo trials amongst patients with advanced lung cancer and identify several implications for future research with and care for patients with advanced cancer.

TRIAL REGISTRATION

ISRCTN80812769 . Registered on 8 July 2005.

[Propensity score: A credible alternative to randomization?]

In clinical research, the reference method to evaluate treatment benefit without bias is the randomized trial. Unfortunately, it is not always possible to realize one, as for example in surgery or for particular observational studies. In these cases, Rosenbaum and Rubin introduced in 1983 a new methodology: the calculation of a propensity score. When several treatments are compared, this calculation enables to take into account confusion bias using a score that synthesizes the influence on treatment choice of clinical parameters evaluated before. This article describes how to build this score, to estimate its validity, and how to use it: as a new variable into a multivariate analysis, as a matching criterion, or as a stratification parameter. Examples are given to illustrate each case and point out the limitations of such a methodology. This approach, although innovative and useful, cannot reach the level of evidence of randomized clinical trials: simulations have demonstrated this fact in several situations. On the other hand, it can be compared to standard multivariate analysis which permits in a non-randomized context, to limit evaluation bias of treatments by adjusting on potential confusion factors. Some guidelines are given in the last chapter to help researchers decide whether to use a propensity score or a standard multivariate analysis.

[Evaluating the efficacy of a new drug]

AIM

To give a practical guide for phase III clinical trial reading in order to evaluate independently the efficacy of a (new) drug therapy.

METHOD

Synthesis established by public health teachers and illustrated with examples in the field of urology.

RESULTS

A structured and simplified critical review of clinical trial remains the main way to "have an idea" of a new drug efficacy. Efficacy in clinical trial may be far from patient satisfaction, willingness to pursue with this treatment or cost-effectiveness relevance.

CONCLUSION

Knowledge of patient expectation, disease evolution and epidemiology have to be put side by side to raw efficacy data coming from phase III study in order to evaluate efficiency and effectiveness of new drugs therapy.