Development process of a consensus-driven CONSORT extension for randomised trials using an adaptive design

Who should value children's health and how? An international Delphi study

Valuing child health necessitates normative methodological decisions on whose preferences should be elicited and who should be imagined as experiencing impaired health. Formal guidance is limited and expert consensus unclear. This study sought to establish the degree of consensus among expert stakeholders on normative issues of who to ask and who should be imagined when valuing child health (7-17 years) to inform UK health technology assessment. Sixty-two experts (n = 47 in Round 2) from 18 countries participated in a modified, two-round online Delphi survey (Round 1: May-June 2023; Round 2: September-October 2023). Participants were expert stakeholders in child health valuation, including academics (n = 38); industry/consultancy representatives (including the charity/not-for-profit sector; n = 13); and UK policy/government representatives (n = 11). The Delphi survey was modified between rounds and consisted of 9-point Likert, categorical, multiple-choice, and free-text questions on normative issues in valuing child health. Responses were analysed descriptively and thematically. An a priori criterion of ≥75% agreement was established for formal consensus, while areas approaching consensus (≥70% agreement) and without consensus were identified as a future research primer. Consensus was observed that older adolescents (aged 16-17 years) and adults (18+ years) should be asked to value child health states. There was consensus that the former should think about themselves when valuing the health states and the latter should imagine a child of some form (e.g., imagining themselves as a child or another hypothetical child). However, no consensus was evident on what form this should take. Several other methodological issues also reached consensus. These findings are largely consistent with recent views elicited qualitatively from members of the public and other stakeholders on normative issues in valuing child health. The results mean that, contrary to what has been done in previous child health valuation studies, efforts should be made to involve both older adolescents (16+ years) and adults in child health valuation.

Sample size in multistakeholder Delphi surveys: at what minimum sample size do replicability of results stabilize?

BACKGROUND AND OBJECTIVE

The minimum sample size for multistakeholder Delphi surveys remains understudied. Drawing from three large international multistakeholder Delphi surveys, this study aimed to: 1) investigate the effect of increasing sample size on replicability of results; 2) assess whether the level of replicability of results differed with participant characteristics: for example, gender, age, and profession.

METHODS

We used data from Delphi surveys to develop guidance for improved reporting of health-care intervention trials: SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) and CONSORT (Consolidated Standards of Reporting Trials) extension for surrogate end points (n = 175, 22 items rated); CONSORT-SPI [CONSORT extension for Social and Psychological Interventions] (n = 333, 77 items rated); and core outcome set for burn care (n = 553, 88 items rated). Resampling with replacement was used to draw random subsamples from the participant data set in each of the three surveys. For each subsample, the median value of all rated survey items was calculated and compared to the medians from the full participant data set. The median number (and interquartile range) of medians replicated was used to calculate the percentage replicability (and variability). High replicability was defined as ≥80% and moderate as 60% and <80% RESULTS: The average median replicability (variability) as a percentage of total number of items rated from the three datasets was 81% (10%) at a sample size of 60. In one of the datasets (CONSORT-SPI), a ≥80% replicability was reached at a sample size of 80. On average, increasing the sample size from 80 to 160 increased the replicability of results by a further 3% and reduced variability by 1%. For subgroup analysis based on participant characteristics (eg, gender, age, professional role), using resampled samples of 20 to 100 showed that a sample size of 20 to 30 resulted to moderate replicability levels of 64% to 77%.

CONCLUSION

We found that a minimum sample size of 60-80 participants in multistakeholder Delphi surveys provides a high level of replicability (≥80%) in the results. For Delphi studies limited to individual stakeholder groups (such as researchers, clinicians, patients), a sample size of 20 to 30 per group may be sufficient.

Methods used to develop the SPIRIT 2024 and CONSORT 2024 Statements

OBJECTIVES

To describe, and explain the rationale for, the methods used and decisions made during development of the updated SPIRIT 2024 and CONSORT 2024 reporting guidelines.

METHODS

We developed SPIRIT 2024 and CONSORT 2024 together to facilitate harmonization of the two guidelines, and incorporated content from key extensions. We conducted a scoping review of comments suggesting changes to SPIRIT 2013 and CONSORT 2010, and compiled a list of other possible revisions based on existing SPIRIT and CONSORT extensions, other reporting guidelines, and personal communications. From this, we generated a list of potential modifications or additions to SPIRIT and CONSORT, which we presented to stakeholders for feedback in an international online Delphi survey. The Delphi survey results were discussed at an online expert consensus meeting attended by 30 invited international participants. We then drafted the updated SPIRIT and CONSORT checklists and revised them based on further feedback from meeting attendees.

RESULTS

We compiled 83 suggestions for revisions or additions to SPIRIT and/or CONSORT from the scoping review and 85 from other sources, from which we generated 33 potential changes to SPIRIT (n = 5) or CONSORT (n = 28). Of 463 participants invited to take part in the Delphi survey, 317 (68%) responded to Round 1, 303 (65%) to Round 2 and 290 (63%) to Round 3. Two additional potential checklist changes were added to the Delphi survey based on Round 1 comments. Overall, 14/35 (SPIRIT n = 0; CONSORT n = 14) proposed changes reached the predefined consensus threshold (≥80% agreement), and participants provided 3580 free-text comments. The consensus meeting participants agreed with implementing 11/14 of the proposed changes that reached consensus in the Delphi and supported implementing a further 4/21 changes (SPIRIT n = 2; CONSORT n = 2) that had not reached the Delphi threshold. They also recommended further changes to refine key concepts and for clarity.

CONCLUSION

The forthcoming SPIRIT 2024 and CONSORT 2024 Statements will provide updated, harmonized guidance for reporting randomized controlled trial protocols and results, respectively. The simultaneous development of the SPIRIT and CONSORT checklists has been informed by current empirical evidence and extensive input from stakeholders. We hope that this report of the methods used will be helpful for developers of future reporting guidelines.

Advice only versus advice and a physiotherapy programme for acute traumatic anterior shoulder dislocation: the ARTISAN RCT

Background

The extra benefit of a programme of physiotherapy in addition to advice alone, following first-time traumatic shoulder dislocation, is uncertain. We compared the clinical and cost-effectiveness of a single session of advice with a single session of advice and a programme of physiotherapy.

Objective

The primary objective was to quantify and draw inferences about observed differences in the Oxford Shoulder Instability Score between the trial treatment groups 6 months post randomisation, in adults with a first-time traumatic shoulder dislocation.

Design

A pragmatic, multicentre, superiority, randomised controlled trial with embedded qualitative study.

Setting

Forty-one hospitals in the UK NHS.

Participants

Adults with a radiologically confirmed first-time traumatic anterior shoulder dislocation, being managed non-operatively. People with neurovascular complications or bilateral dislocations, and those unable to adhere to trial procedures or unable to attend physiotherapy within 6 weeks of injury, or who had previously been randomised, were excluded.

Interventions

All participants received the same initial shoulder examination followed by advice to aid self-management, lasting up to 1 hour and administered by a physiotherapist (control). Participants randomised to receive an additional programme of physiotherapy were offered sessions lasting for up to 30 minutes, over a maximum duration of 4 months from the date of randomisation (intervention).

Main outcome measures

The primary outcome measure was the Oxford Shoulder Instability Score. This is a self-completed outcome measure containing 12 questions (0-4 points each), with possible scores from 0 (worst function) to 48 (best function). Measurements were collected at 6 weeks, 3 months, 6 months and 12 months by postal questionnaire; 6 months was the primary outcome time point. The primary health outcome for economic evaluation was the quality-adjusted life-year, in accordance with National Institute of Health and Care Excellence guidelines.

Results

Between 14 November 2018 and 14 March 2022, 482 participants were randomised to advice (n = 240) or advice and a programme of physiotherapy (n = 242). Participants were 34% female, with a mean age of 45 years, and treatment arms were balanced at baseline. There was not a statistically significant difference in the primary outcome between advice only and advice plus a programme of physiotherapy at 6 months for the primary intention-to-treat adjusted analysis (favours physiotherapy: 1.5, 95% confidence interval -0.3 to 3.5) or at earlier 3-month and 6-week time points on the Oxford Shoulder Instability Score (0-48; higher scores indicate better function). The probability of physiotherapy being cost-effective at a willingness-to-pay threshold of £30,000 was 0.95.

Conclusions

We found little difference in the primary outcome or other secondary outcomes. Advice with additional physiotherapy sessions was found likely to be cost-effective. However, small imprecise incremental costs and quality-adjusted life-years raise questions on whether it is the best use of scarce physiotherapy resources given current service demands.

Limitations

Loss to follow-up was 27%; however, the observed standard deviation was much smaller than anticipated. These changes in parameters reduced the number of participants required to observe the planned target difference of four points. Our post hoc sensitivity analysis, accounting for missing data, gives similar results.

Future work

Further research should be directed towards optimising self-management strategies.

Study registration

This study is registered as ISRCTN63184243.

Funding

This award was funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme (NIHR award ref: 16/167/56) and is published in full in Health Technology Assessment; Vol. 28, No. 22. See the NIHR Funding and Awards website for further award information.

Long-term health outcomes for patients with obstructive sleep apnea: placing the Agency for Healthcare Research and Quality report in context-a multisociety commentary

This multisociety commentary critically examines the Agency for Healthcare Research and Quality (AHRQ) final report and systematic review on long-term health outcomes in obstructive sleep apnea. The AHRQ report was commissioned by the Centers for Medicare & Medicaid Services and particularly focused on the long-term patient-centered outcomes of continuous positive airway pressure, the variability of sleep-disordered breathing metrics, and the validity of these metrics as surrogate outcomes. This commentary raises concerns regarding the AHRQ report conclusions and their potential implications for policy decisions. A major concern expressed in this commentary is that the AHRQ report inadequately acknowledges the benefits of continuous positive airway pressure for several established, long-term clinically important outcomes including excessive sleepiness, motor vehicle accidents, and blood pressure. While acknowledging the limited evidence for the long-term benefits of continuous positive airway pressure treatment, especially cardiovascular outcomes, as summarized by the AHRQ report, this commentary reviews the limitations of recent randomized controlled trials and nonrandomized controlled studies and the challenges of conducting future randomized controlled trials. A research agenda to address these challenges is proposed including study designs that may include both high quality randomized controlled trials and nonrandomized controlled studies. This commentary concludes by highlighting implications for the safety and quality of life for the millions of people living with obstructive sleep apnea if the AHRQ report alone was used by payers to limit coverage for the treatment of obstructive sleep apnea while not considering the totality of available evidence.

CITATION

Patil SP, Billings ME, Bourjeily G, et al. Long-term health outcomes for patients with obstructive sleep apnea: placing the Agency for Healthcare Research and Quality report in context-a multisociety commentary. J Clin Sleep Med. 2024;20(1):135-149.

Reporting quality was suboptimal in a systematic review of randomized controlled trials with adaptive designs

OBJECTIVE

The study was conducted to evaluate the reporting quality of randomized controlled trials (RCTs) that use an adaptive design (AD) based on the 2020 AD Consolidated Standards for Reporting Trials 2010 extension (ACE) guidelines and identify factors associated with better reporting quality.

STUDY DESIGN AND SETTING

PubMed, Embase, Cochrane, Web of Science, and Google Scholar were searched until November 1, 2022. Multivariable linear regression analysis was performed to investigate potential predictors.

RESULTS

In total, 109 RCTs were included in our study. The mean compliance rate for the ACE checklist items was 69.75% ± 16.02. Key methodological items including allocation concealment and its implementations were poorly reported. There was also suboptimal reporting of checklist items related to the conduct of interim analyzes. Multivariable regression analysis showed better reporting quality with trial registration, nonindustry affiliation (first author), a sample size of >100, general medical journal type, publication date (≥2020), funding, and disclosure of the number of interim analyzes.

CONCLUSION

Our study showed that RCTs with AD had suboptimal reporting of 2020 ACE checklist items, particularly AD-specific items. Following the development of ACE guidelines, stricter adherence to the ACE guideline is necessary to improve their reporting quality. Pre-ACE and post-ACE adherence comparisons can be conducted in the future.

Protocol for the development of SPIRIT and CONSORT extensions for randomised controlled trials with surrogate primary endpoints: SPIRIT-SURROGATE and CONSORT-SURROGATE

INTRODUCTION

Randomised controlled trials (RCTs) may use surrogate endpoints as substitutes and predictors of patient-relevant/participant-relevant final outcomes (eg, survival, health-related quality of life). Translation of effects measured on a surrogate endpoint into health benefits for patients/participants is dependent on the validity of the surrogate; hence, more accurate and transparent reporting on surrogate endpoints is needed to limit misleading interpretation of trial findings. However, there is currently no explicit guidance for the reporting of such trials. Therefore, we aim to develop extensions to the SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) and CONSORT (Consolidated Standards of Reporting Trials) reporting guidelines to improve the design and completeness of reporting of RCTs and their protocols using a surrogate endpoint as a primary outcome.

METHODS AND ANALYSIS

The project will have four phases: phase 1 (literature reviews) to identify candidate reporting items to be rated in a Delphi study; phase 2 (Delphi study) to rate the importance of items identified in phase 1 and receive suggestions for additional items; phase 3 (consensus meeting) to agree on final set of items for inclusion in the extensions and phase 4 (knowledge translation) to engage stakeholders and disseminate the project outputs through various strategies including peer-reviewed publications. Patient and public involvement will be embedded into all project phases.

ETHICS AND DISSEMINATION

The study has received ethical approval from the University of Glasgow College of Medical, Veterinary and Life Sciences Ethics Committee (project no: 200210051). The findings will be published in open-access peer-reviewed publications and presented in conferences, meetings and relevant forums.

Controlled evaLuation of Angiotensin Receptor Blockers for COVID-19 respIraTorY disease (CLARITY): statistical analysis plan for a randomised controlled Bayesian adaptive sample size trial

The CLARITY trial (Controlled evaLuation of Angiotensin Receptor Blockers for COVID-19 respIraTorY disease) is a two-arm, multi-centre, randomised controlled trial being run in India and Australia that investigates the effectiveness of angiotensin receptor blockers in addition to standard care compared to placebo (in Indian sites) with standard care in reducing the duration and severity of lung failure in patients with COVID-19. The trial was designed as a Bayesian adaptive sample size trial with regular planned analyses where pre-specified decision rules will be assessed to determine whether the trial should be stopped due to sufficient evidence of treatment effectiveness or futility. Here, we describe the statistical analysis plan for the trial and define the pre-specified decision rules, including those that could lead to the trial being halted. The primary outcome is clinical status on a 7-point ordinal scale adapted from the WHO Clinical Progression scale assessed at day 14. The primary analysis will follow the intention-to-treat principle. A Bayesian adaptive trial design was selected because there is considerable uncertainty about the extent of potential benefit of this treatment.Trial registrationClinicalTrials.gov NCT04394117 . Registered on 19 May 2020Clinical Trial Registry of India CTRI/2020/07/026831Version and revisionsVersion 1.0. No revisions.

Updating guidance for reporting systematic reviews: development of the PRISMA 2020 statement

OBJECTIVES

To describe the processes used to update the PRISMA 2009 statement for reporting systematic reviews, present results of a survey conducted to inform the update, summarize decisions made at the PRISMA update meeting, and describe and justify changes made to the guideline.

METHODS

We reviewed 60 documents with reporting guidance for systematic reviews to generate suggested modifications to the PRISMA 2009 statement. We invited 220 systematic review methodologists and journal editors to complete a survey about the suggested modifications. The results of these projects were discussed at a 21-member in-person meeting. Following the meeting, we drafted the PRISMA 2020 statement and refined it based on feedback from co-authors and a convenience sample of 15 systematic reviewers.

RESULTS

The review of 60 documents revealed that all topics addressed by the PRISMA 2009 statement could be modified. Of the 110 survey respondents, more than 66% recommended keeping six of the original checklist items as they were and modifying 15 of them using wording suggested by us. Attendees at the in-person meeting supported the revised wording for several items but suggested rewording for most to enhance clarity, and further refinements were made over six drafts of the guideline.

CONCLUSIONS

The PRISMA 2020 statement consists of updated reporting guidance for systematic reviews. We hope that providing this detailed description of the development process will enhance the acceptance and uptake of the guideline and assist those developing and updating future reporting guidelines.

Facing the urgency of therapies for progressive MS - a Progressive MS Alliance proposal

Therapies for infiltrative inflammation in multiple sclerosis (MS) have advanced greatly, but neurodegeneration and compartmentalized inflammation remain virtually untargeted as in other diseases of the nervous system. Consequently, many therapies are available for the relapsing-remitting form of MS, but the progressive forms remain essentially untreated. The objective of the International Progressive MS Alliance is to expedite the development of effective therapies for progressive MS through new initiatives that foster innovative thinking and concrete advancements. Based on these principles, the Alliance is developing a new funding programme that will focus on experimental medicine trials. Here, we discuss the reasons behind the focus on experimental medicine trials, the strengths and weaknesses of these approaches and of the programme, and why we hope to advance therapies while improving the understanding of progression in MS. We are soliciting public and academic feedback, which will help shape the programme and future strategies of the Alliance.

Utilising benefit-risk assessments within clinical trials-a protocol for the BRAINS project

BACKGROUND

Depending on the treatment to be investigated, a clinical trial could be designed to assess objectives of superiority, equivalence or non-inferiority. The design of the study is affected by many different elements including the control treatment, the primary outcome and associated relationships. In some studies, there could be more than one outcome of interest. In these situations, benefit-risk methodologies could be used to assess the outcomes simultaneously and consider the trade-off between the benefits against the risks of a treatment. Benefit-risk is used within the regulatory industry but seldom included within publicly funded clinical trials within the UK. This project aims to gain an expert consensus on how to select the appropriate trial design (e.g. superiority) and when to consider including benefit-risk methods.

METHODS

The project will consist of four work packages: 1. A web-based survey to elicit current experiences and opinions, 2. A rapid literature review to assess any current recommendations, 3. A two-day consensus workshop to gain agreement on the recommendations, and 4. Production of a guidance document.

DISCUSSION

The aim of the project is to provide a guideline for clinical researchers, grant funding bodies and reviewers for grant bodies for how to select the most appropriate trial design and when it is appropriate to consider using benefit-risk methods. The focus of the guideline will be on publicly funded trials however, the vision is that the work will be applicable across research settings and we will connect with other organisations and committees as appropriate.

Adding flexibility to clinical trial designs: an example-based guide to the practical use of adaptive designs

Adaptive designs for clinical trials permit alterations to a study in response to accumulating data in order to make trials more flexible, ethical, and efficient. These benefits are achieved while preserving the integrity and validity of the trial, through the pre-specification and proper adjustment for the possible alterations during the course of the trial. Despite much research in the statistical literature highlighting the potential advantages of adaptive designs over traditional fixed designs, the uptake of such methods in clinical research has been slow. One major reason for this is that different adaptations to trial designs, as well as their advantages and limitations, remain unfamiliar to large parts of the clinical community. The aim of this paper is to clarify where adaptive designs can be used to address specific questions of scientific interest; we introduce the main features of adaptive designs and commonly used terminology, highlighting their utility and pitfalls, and illustrate their use through case studies of adaptive trials ranging from early-phase dose escalation to confirmatory phase III studies.

Pragmatic, adaptive clinical trials: Is 2020 the dawning of a new age?

Given the high case fatality rate of SARS-CoV-2, for which there is no cure and no vaccine, clinicians are forced to make decisions about how best to manage patients with limited high-quality evidence to guide treatment. Traditional randomized controlled trials provide strong experimental evidence, however, tend to be slow, inflexible, and have limited generalizability. Adaptive and pragmatic designs are an attractive alternative, which meet our ethical obligation during the SARS-CoV-2 pandemic to balance speed, agility, and generalizability with both prospective study and scientific rigor.

The ORVAC trial: a phase IV, double-blind, randomised, placebo-controlled clinical trial of a third scheduled dose of Rotarix rotavirus vaccine in Australian Indigenous infants to improve protection against gastroenteritis: a statistical analysis plan

Objective

The purpose of this double-blind, randomised, placebo-controlled, adaptive design trial with frequent interim analyses is to determine if Australian Indigenous children, who receive an additional (third) dose of human rotavirus vaccine (Rotarix, GlaxoSmithKline) for children aged 6 to < 12 months, would improve protection against clinically significant all-cause gastroenteritis.

Participants

Up to 1000 Australian Aboriginal and Torres Strait Islander (hereafter Indigenous) infants aged 6 to < 12 months will be recruited from all regions of the Northern Territory.

Interventions

The intervention is the addition of a third scheduled dose of human monovalent rotavirus vaccine.

Co-primary and secondary outcome measures

ORVAC has two co-primary outcomes: (1) anti-rotavirus IgA seroconversion, defined as serum anti-rotavirus IgA ≥ 20 U/ml 28 to 55 days post Rotarix/placebo, and (2) time from randomisation to medical attendance for which the primary reason for presentation is acute gastroenteritis or acute diarrhoea illness before age 36 months. Secondary outcomes include (1) change in anti-rotavirus IgA log titre, (2) time from randomisation to hospitalisation with primary admission code presumed or confirmed acute diarrhoea illness before age 36 months, (3) time from randomisation to hospitalisation for which the admission is rotavirus confirmed diarrhoea illness before age 36 months and (4) time from randomisation to rotavirus infection (not necessarily requiring hospitalisation) meeting the jurisdictional definition before age 36 months.

Discussion

A detailed, prospective statistical analysis plan is presented for this Bayesian adaptive design. The plan was written by the trial statistician and details the study design, pre-specified adaptative elements, decision thresholds, statistical methods and the simulations used to evaluate the operating characteristics of the trial. As at August 2020, four interim analyses have been run, but no stopping rules have been triggered. Application of this SAP will minimise bias and supports transparent and reproducible research.

Trial registration

Clinicaltrials.gov NCT02941107. Registered on 21 October 2016

Original protocol for the study

10.1136/bmjopen-2019-032549

The adaptive designs CONSORT extension (ACE) statement: a checklist with explanation and elaboration guideline for reporting randomised trials that use an adaptive design

Adaptive designs (ADs) allow pre-planned changes to an ongoing trial without compromising the validity of conclusions and it is essential to distinguish pre-planned from unplanned changes that may also occur. The reporting of ADs in randomised trials is inconsistent and needs improving. Incompletely reported AD randomised trials are difficult to reproduce and are hard to interpret and synthesise. This consequently hampers their ability to inform practice as well as future research and contributes to research waste. Better transparency and adequate reporting will enable the potential benefits of ADs to be realised.This extension to the Consolidated Standards Of Reporting Trials (CONSORT) 2010 statement was developed to enhance the reporting of randomised AD clinical trials. We developed an Adaptive designs CONSORT Extension (ACE) guideline through a two-stage Delphi process with input from multidisciplinary key stakeholders in clinical trials research in the public and private sectors from 21 countries, followed by a consensus meeting. Members of the CONSORT Group were involved during the development process.The paper presents the ACE checklists for AD randomised trial reports and abstracts, as well as an explanation with examples to aid the application of the guideline. The ACE checklist comprises seven new items, nine modified items, six unchanged items for which additional explanatory text clarifies further considerations for ADs, and 20 unchanged items not requiring further explanatory text. The ACE abstract checklist has one new item, one modified item, one unchanged item with additional explanatory text for ADs, and 15 unchanged items not requiring further explanatory text.The intention is to enhance transparency and improve reporting of AD randomised trials to improve the interpretability of their results and reproducibility of their methods, results and inference. We also hope indirectly to facilitate the much-needed knowledge transfer of innovative trial designs to maximise their potential benefits. In order to encourage its wide dissemination this article is freely accessible on the BMJ and Trials journal websites."To maximise the benefit to society, you need to not just do research but do it well" Douglas G Altman.

The Adaptive designs CONSORT Extension (ACE) statement: a checklist with explanation and elaboration guideline for reporting randomised trials that use an adaptive design

Adaptive designs (ADs) allow pre-planned changes to an ongoing trial without compromising the validity of conclusions and it is essential to distinguish pre-planned from unplanned changes that may also occur. The reporting of ADs in randomised trials is inconsistent and needs improving. Incompletely reported AD randomised trials are difficult to reproduce and are hard to interpret and synthesise. This consequently hampers their ability to inform practice as well as future research and contributes to research waste. Better transparency and adequate reporting will enable the potential benefits of ADs to be realised.This extension to the Consolidated Standards Of Reporting Trials (CONSORT) 2010 statement was developed to enhance the reporting of randomised AD clinical trials. We developed an Adaptive designs CONSORT Extension (ACE) guideline through a two-stage Delphi process with input from multidisciplinary key stakeholders in clinical trials research in the public and private sectors from 21 countries, followed by a consensus meeting. Members of the CONSORT Group were involved during the development process.The paper presents the ACE checklists for AD randomised trial reports and abstracts, as well as an explanation with examples to aid the application of the guideline. The ACE checklist comprises seven new items, nine modified items, six unchanged items for which additional explanatory text clarifies further considerations for ADs, and 20 unchanged items not requiring further explanatory text. The ACE abstract checklist has one new item, one modified item, one unchanged item with additional explanatory text for ADs, and 15 unchanged items not requiring further explanatory text.The intention is to enhance transparency and improve reporting of AD randomised trials to improve the interpretability of their results and reproducibility of their methods, results and inference. We also hope indirectly to facilitate the much-needed knowledge transfer of innovative trial designs to maximise their potential benefits.

Evaluation of the efficacy and safety of intravenous remdesivir in adult patients with severe COVID-19: study protocol for a phase 3 randomized, double-blind, placebo-controlled, multicentre trial

Background

Coronavirus disease 2019 (COVID-19), caused by a novel corinavirus (later named SARS-CoV-2 virus), was fistly reported in Wuhan, Hubei Province, China towards the end of 2019. Large-scale spread within China and internationally led the World Health Organization to declare a Public Health Emergency of International Concern on 30^th January 2020. The clinical manifestations of COVID-19 virus infection include asymptomatic infection, mild upper respiratory symptoms, severe viral pneumonia with respiratory failure, and even death. There are no antivirals of proven clinical efficacy in coronavirus infections. Remdesivir (GS-5734), a nucleoside analogue, has inhibitory effects on animal and human highly pathogenic coronaviruses, including MERS-CoV and SARS-CoV, in in vitro and in vivo experiments. It is also inhibitory against the COVID-19 virus in vitro. The aim of this study is to assess the efficacy and safety of remdesivir in adult patients with severe COVID-19.

Methods

The protocol is prepared in accordance with the SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) guidelines. This is a phase 3, randomized, double-blind, placebo-controlled, multicentre trial. Adults (≥ 18 years) with laboratory-confirmed COVID-19 virus infection, severe pneumonia signs or symptoms, and radiologically confirmed severe pneumonia are randomly assigned in a 2:1 ratio to intravenously administered remdesivir or placebo for 10 days. The primary endpoint is time to clinical improvement (censored at day 28), defined as the time (in days) from randomization of study treatment (remdesivir or placebo) until a decline of two categories on a six-category ordinal scale of clinical status (1 = discharged; 6 = death) or live discharge from hospital. One interim analysis for efficacy and futility will be conducted once half of the total number of events required has been observed.

Discussion

This is the first randomized, placebo-controlled trial in COVID-19. Enrolment began in sites in Wuhan, Hubei Province, China on 6^th February 2020.

Trial registration

ClinicalTrials.gov: NCT04257656. Registered on 6 February 2020.

Protocol for a randomised controlled trial of Subacromial spacer for Tears Affecting Rotator cuff Tendons: a Randomised, Efficient, Adaptive Clinical Trial in Surgery (START:REACTS)

INTRODUCTION

Shoulder pain due to irreparable rotator cuff tears can cause substantial disability, but treatment options are limited. A balloon spacer is a relatively simple addition to a standard arthroscopic debridement procedure, but it is costly and there is no current randomised trial evidence to support its use. This trial will evaluate the clinical and cost-effectiveness of a subacromial balloon spacer for individuals undergoing arthroscopic debridement for irreparable rotator cuff tears.New surgical procedures can provide substantial benefit to patients. Good quality randomised controlled trials (RCTs) are needed, but trials in surgery are typically long and expensive, exposing patients to risk and the healthcare system to substantial costs. One way to improve the efficiency of trials is with an adaptive sample size. Such methods are well established in drug trials but have rarely, if ever, been used in surgical trials.

METHODS AND ANALYSIS

Subacromial spacer for Tears Affecting Rotator cuff Tendons: a Randomised, Efficient, Adaptive Clinical Trial in Surgery (START:REACTS) is a participant and assessor blinded, adaptive, multicentre RCT comparing arthroscopic debridement with the InSpace balloon (Stryker, USA) to arthroscopic debridement alone for people with a symptomatic irreparable rotator cuff tear. It uses a group sequential adaptive design where interim analyses are performed using all of the 3, 6 and 12-month data that are available at each time point. A maximum of 221 participants will be randomised (1:1 ratio), this will provide 90% power (at the 5% level) for a 6 point difference in the primary outcome; the Oxford Shoulder Score at 12 months. A substudy will use deltoid-active MRI scans in 56 participants to assess the function of the balloon. Analysis will be on an intention-to-treat basis and reported according to principles established in the Consolidated Standards of Reporting Trials statement.

ETHICS AND DISSEMINATION

NRES number 18/WM/0025. The results will be disseminated via peer-reviewed publications, presentations at conferences, lay summaries and social media.

TRIAL REGISTRATION NUMBER

ISRCTN17825590.

An overview of platform trials with a checklist for clinical readers

OBJECTIVES

The objective of the study was to outline key considerations for general clinical readers when critically evaluating publications on platform trials and for researchers when designing these types of clinical trials.

STUDY DESIGN AND SETTING

In this review, we describe key concepts of platform trials with case study discussion of two hallmark platform trials in STAMPEDE and I-SPY2. We provide reader's guide to platform trials with a critical appraisal checklist.

RESULTS

Platform trials offer flexibilities of dropping ineffective arms early based on interim data and introducing new arms into the trial. For platform trials, it is important to consider how interventions are compared and evaluated throughout and how new interventions are introduced. For intervention comparisons, it is important to consider what the primary analysis is, what and how many interventions are active simultaneously, and allocation between different arms. Interim evaluation considerations should include the number and timing of interim evaluations and outcomes and statistical rules used to drop interventions. New interventions are usually introduced based on scientific merits, so consideration of these merits is important, together with the timing and mechanisms in which new interventions are added.

CONCLUSION

More efforts are needed to improve the scientific literacy of platform trials. Our review provides an overview of the important concepts of platform trials.

How can health economics be used in the design and analysis of adaptive clinical trials? A qualitative analysis

INTRODUCTION

Adaptive designs offer a flexible approach, allowing changes to a trial based on examinations of the data as it progresses. Adaptive clinical trials are becoming a popular choice, as the prudent use of finite research budgets and accurate decision-making are priorities for healthcare providers around the world. The methods of health economics, which aim to maximise the health gained for money spent, could be incorporated into the design and analysis of adaptive clinical trials to make them more efficient. We aimed to understand the perspectives of stakeholders in health technology assessments to inform recommendations for the use of health economics in adaptive clinical trials.

METHODS

A qualitative study explored the attitudes of key stakeholders-including researchers, decision-makers and members of the public-towards the use of health economics in the design and analysis of adaptive clinical trials. Data were collected using interviews and focus groups (29 participants). A framework analysis was used to identify themes in the transcripts.

RESULTS

It was considered that answering the clinical research question should be the priority in a clinical trial, notwithstanding the importance of cost-effectiveness for decision-making. Concerns raised by participants included handling the volatile nature of cost data at interim analyses; implementing this approach in global trials; resourcing adaptive trials which are designed and adapted based on health economic outcomes; and training stakeholders in these methods so that they can be implemented and appropriately interpreted.

CONCLUSION

The use of health economics in the design and analysis of adaptive clinical trials has the potential to increase the efficiency of health technology assessments worldwide. Recommendations are made concerning the development of methods allowing the use of health economics in adaptive clinical trials, and suggestions are given to facilitate their implementation in practice.

A review of available software for adaptive clinical trial design

BACKGROUND/AIMS

The increasing cost of the drug development process has seen interest in the use of adaptive trial designs grow substantially. Accordingly, much research has been conducted to identify barriers to increasing the use of adaptive designs in practice. Several articles have argued that the availability of user-friendly software will be an important step in making adaptive designs easier to implement. Therefore, we present a review of the current state of software availability for adaptive trial design.

METHODS

We review articles from 31 journals published in 2013-2017 that relate to methodology for adaptive trials to assess how often code and software for implementing novel adaptive designs is made available at the time of publication. We contrast our findings against these journals' policies on code distribution. We also search popular code repositories, such as Comprehensive R Archive Network and GitHub, to identify further existing user-contributed software for adaptive designs. From this, we are able to direct interested parties toward solutions for their problem of interest.

RESULTS

Only 30% of included articles made their code available in some form. In many instances, articles published in journals that had mandatory requirements on code provision still did not make code available. There are several areas in which available software is currently limited or saturated. In particular, many packages are available to address group sequential design, but comparatively little code is present in the public domain to determine biomarker-guided adaptive designs.

CONCLUSIONS

There is much room for improvement in the provision of software alongside adaptive design publications. In addition, while progress has been made, well-established software for various types of trial adaptation remains sparsely available.

A web application for the design of multi-arm clinical trials

BACKGROUND

Multi-arm designs provide an effective means of evaluating several treatments within the same clinical trial. Given the large number of treatments now available for testing in many disease areas, it has been argued that their utilisation should increase. However, for any given clinical trial there are numerous possible multi-arm designs that could be used, and choosing between them can be a difficult task. This task is complicated further by a lack of available easy-to-use software for designing multi-arm trials.

RESULTS

To aid the wider implementation of multi-arm clinical trial designs, we have developed a web application for sample size calculation when using a variety of popular multiple comparison corrections. Furthermore, the application supports sample size calculation to control several varieties of power, as well as the determination of optimised arm-wise allocation ratios. It is built using the Shiny package in the R programming language, is free to access on any device with an internet browser, and requires no programming knowledge to use. It incorporates a variety of features to make it easier to use, including help boxes and warning messages. Using design parameters motivated by a recently completed phase II oncology trial, we demonstrate that the application can effectively determine and evaluate complex multi-arm trial designs.

CONCLUSIONS

The application provides the core information required by statisticians and clinicians to review the operating characteristics of a chosen multi-arm clinical trial design. The range of designs supported by the application is broader than other currently available software solutions. Its primary limitation, particularly from a regulatory agency point of view, is its lack of validation. However, we present an approach to efficiently confirming its results via simulation.

A systematic survey of randomised trials that stopped early for reasons of futility

Background

Randomised trial protocols may incorporate interim analyses, with the potential to stop the study for futility if early data show insufficient promise of a treatment benefit. Previously, we have shown that this approach will theoretically lead to mis-estimation of the treatment effect. We now wished to ascertain the importance of this phenomenon in practice.

Methods

We reviewed the methods and results in a set of trials that had stopped for futility, identified through an extensive literature search. We recorded clinical areas, interventions, study design, outcomes, trial setting, sponsorship, planned and actual treatment effects, sample sizes; power; and if there was a data safety monitoring board, or a published protocol. We identified: if interim analyses were pre-specified, and how many analyses actually occurred; what pre-specified criteria might define futility; if a futility analysis formed the basis for stopping; who made the decision to stop; and the conditional power of each study, i.e. the probability of statistically significant results if the study were to continue to its complete sample size.

Results

We identified 52 eligible trials, covering many clinical areas. Most trials had multiple centres, tested drugs, and 40% were industry sponsored. There were 75% where at least one interim analysis was planned a priori; a majority had only one interim analysis, typically with about half the target total sample size. A majority of trials did not pre-define a stopping rule, and a variety of reasons were given for stopping. Few studies calculated and reported low conditional power to justify the early stop. When conditional power could be calculated, it was typically low, especially under the current trend hypothesis. However, under the original design hypothesis, a few studies had relatively high conditional power. Data collection often continued after the interim analysis.

Conclusions

Although other factors will typically be involved, we conclude that, from the perspective of conditional power, stopping early for futility was probably reasonable in most cases, but documentation of the basis for stopping was often missing or vague. Interpretation of truncated trials would be enhanced by improved reporting of stopping protocols, and of their actual execution.

Bayesian adaptive designs for multi-arm trials: an orthopaedic case study

BACKGROUND

Bayesian adaptive designs can be more efficient than traditional methods for multi-arm randomised controlled trials. The aim of this work was to demonstrate how Bayesian adaptive designs can be constructed for multi-arm phase III clinical trials and assess potential benefits that these designs offer.

METHODS

We constructed several alternative Bayesian adaptive designs for the Collaborative Ankle Support Trial (CAST), which was a randomised controlled trial that compared four treatments for severe ankle sprain. These designs incorporated response adaptive randomisation (RAR), arm dropping, and early stopping for efficacy or futility. We studied the operating characteristics of the Bayesian designs via simulation. We then virtually re-executed the trial by implementing the Bayesian adaptive designs using patient data sampled from the CAST study to demonstrate the practical applicability of the designs.

RESULTS

We constructed five Bayesian adaptive designs, each of which had high power and recruited fewer patients on average than the original designs target sample size. The virtual executions showed that most of the Bayesian designs would have led to trials that declared superiority of one of the interventions over the control. Bayesian adaptive designs with RAR or arm dropping were more likely to allocate patients to better performing arms at each interim analysis. Similar estimates and conclusions were obtained from the Bayesian adaptive designs as from the original trial.

CONCLUSIONS

Using CAST as an example, this case study shows how Bayesian adaptive designs can be constructed for phase III multi-arm trials using clinically relevant decision criteria. These designs demonstrated that they can potentially generate earlier results and allocate more patients to better performing arms. We recommend the wider use of Bayesian adaptive approaches in phase III clinical trials.

TRIAL REGISTRATION

CAST study registration ISRCTN, ISRCTN37807450. Retrospectively registered on 25 April 2003.

Non-inferiority versus superiority trial design for new antibiotics in an era of high antimicrobial resistance: the case for post-marketing, adaptive randomised controlled trials

Antimicrobial resistance is one of the most important threats to global health security. A range of Gram-negative bacteria associated with high morbidity and mortality are now resistant to almost all available antibiotics. In this context of urgency to develop novel drugs, new antibiotics for multidrug-resistant Gram-negative bacteria (namely, ceftazidime-avibactam, plazomicin, and meropenem-vaborbactam) have been approved by regulatory authorities based on non-inferiority trials that provided no direct evidence of their efficacy against multidrug-resistant bacteria such as Enterobacteriaceae spp, Pseudomonas aeruginosa, Stenotrophomonas maltophilia, Burkholderia cepacia, and Acinetobacter baumannii. The use of non-inferiority and superiority trials, and selection of appropriate and optimal study designs, remains a major challenge in the development, registration, and post-marketing implementation of new antibiotics. Using an example of the development process of ceftazidime-avibactam, we propose a strategy for a new research framework based on adaptive randomised clinical trials. The operational research strategy has the aim of assessing the efficacy of new antibiotics in special groups of patients, such as those infected with multidrug-resistant bacteria, who were not included in earlier phase studies, and for whom it is important to establish an appropriate standard of care.