Early phase clinical trials extension to guidelines for the content of statistical analysis plans

Determining a risk-proportionate approach to the validation of statistical programming for clinical trials

BACKGROUND

The contribution of the statistician to the design and analysis of a clinical trial is acknowledged as essential. Ability to reconstruct the statistical contribution to a trial requires rigorous and transparent documentation as evidenced by the reproducibility of results. The process of validating statistical programmes is a key requirement. While guidance relating to software development and life cycle methodologies details steps for validation by information systems developers, there is no guidance applicable to programmes written by statisticians. We aimed to develop a risk-based approach to the validation of statistical programming that would support scientific integrity and efficient resource use within clinical trials units.

METHODS

The project was embedded within the Information Systems Operational Group and the Statistics Operational Group of the UK Clinical Research Collaboration Registered Clinical Trials Unit network. Members were asked to share materials relevant to validation of statistical programming. A review of the published literature, regulatory guidance and knowledge of relevant working groups was undertaken. Surveys targeting the Information Systems Operational Group and Statistics Operational Group were developed to determine current practices across the Registered Clinical Trials Unit network. A risk-based approach was drafted and used as a basis for a workshop with representation from statisticians, information systems developers and quality assurance managers (n = 15). The approach was subsequently modified and presented at a second, larger scale workshop (n = 47) to gain a wider perspective, with discussion of content and implications for delivery. The approach was revised based on the discussions and suggestions made. The workshop was attended by a member of the Medicines for Healthcare products Regulatory Agency Inspectorate who also provided comments on the revised draft.

RESULTS

Types of statistical programming were identified and categorised into six areas: generation of randomisation lists; programmes to explore/understand the data; data cleaning, including complex checks; derivations including data transformations; data monitoring; or interim and final analysis. The risk-based approach considers each category of statistical programme against the impact of an error and its likelihood, whether the programming can be fully prespecified, the need for repeated use and the need for reproducibility. Approaches to the validation of programming within each category are proposed.

CONCLUSION

We have developed a risk-based approach to the validation of statistical programming. It endeavours to facilitate the implementation of targeted quality assurance measures while making efficient use of limited resources.

Statistical methods and data visualisation of patient-reported outcomes in early phase dose-finding oncology trials: a methodological review

Background

Traditionally, within dose-finding clinical trials, treatment toxicity and tolerability are assessed by clinicians. Research has shown that clinician reporting may have inadequate inter-rater reliability, poor correlation with patient reported outcomes, and under capture the true toxicity burden. The introduction of patient-reported outcomes (PROs), where the patient can assess their own symptomatic adverse events or quality of life, has potential to complement current practice to aid dose optimisation. There are no international recommendations offering guidance for the inclusion of PROs in dose-finding trial design and analysis. Our review aimed to identify and describe current statistical methods and data visualisation techniques employed to analyse and visualise PRO data in published early phase dose-finding oncology trials (DFOTs).

Methods

DFOTs published from June 2016-December 2022, which presented PRO analysis methods, were included in this methodological review. We extracted 35 eligible papers indexed in PubMed. Study characteristics extracted included: PRO objectives, PRO measures, statistical analysis and visualisation techniques, and whether the PRO was involved in interim and final dose selection decisions.

Findings

Most papers (30, 85.7%) did not include clear PRO objectives. 20 (57.1%) papers used inferential statistical techniques to analyse PROs, including survival analysis and mixed-effect models. One trial used PROs to classify a clinicians' assessed dose-limiting toxicities (DLTs). Three (8.6%) trials used PROs to confirm the tolerability of the recommended dose. 25 trial reports visually presented PRO data within a figure or table within their publication, of which 12 papers presented PRO score longitudinally.

Interpretation

This review highlighted that the statistical methods and reporting of PRO analysis in DFOTs are often poorly described and inconsistent. Many trials had PRO objectives which were not clearly described, making it challenging to evaluate the appropriateness of the statistical techniques used. Drawing conclusions based on DFOTs which are not powered for PROs may be misleading. With no guidance and standardisation of analysis methods for PROs in early phase DFOTs, it is challenging to compare study findings across trials. Therefore, there is a crucial need to establish international guidance to enhance statistical methods and graphical presentation for PRO analysis in the dose-finding setting.

Funding

EA has been supported to undertake this work as part of a PhD studentship from the Institute of Cancer Research within the MRC/NIHR Trials Methodology Research Partnership. AM is supported by the National Institute for Health Research (NIHR) Biomedical Research Centre at the Royal Marsden NHS Foundation Trust, the Institute of Cancer Research and Imperial College.

Development of consensus-driven SPIRIT and CONSORT extensions for early phase dose-finding trials: the DEFINE study

BACKGROUND

Early phase dose-finding (EPDF) trials are crucial for the development of a new intervention and influence whether it should be investigated in further trials. Guidance exists for clinical trial protocols and completed trial reports in the SPIRIT and CONSORT guidelines, respectively. However, both guidelines and their extensions do not adequately address the characteristics of EPDF trials. Building on the SPIRIT and CONSORT checklists, the DEFINE study aims to develop international consensus-driven guidelines for EPDF trial protocols (SPIRIT-DEFINE) and reports (CONSORT-DEFINE).

METHODS

The initial generation of candidate items was informed by reviewing published EPDF trial reports. The early draft items were refined further through a review of the published and grey literature, analysis of real-world examples, citation and reference searches, and expert recommendations, followed by a two-round modified Delphi process. Patient and public involvement and engagement (PPIE) was pursued concurrently with the quantitative and thematic analysis of Delphi participants' feedback.

RESULTS

The Delphi survey included 79 new or modified SPIRIT-DEFINE (n = 36) and CONSORT-DEFINE (n = 43) extension candidate items. In Round One, 206 interdisciplinary stakeholders from 24 countries voted and 151 stakeholders voted in Round Two. Following Round One feedback, one item for CONSORT-DEFINE was added in Round Two. Of the 80 items, 60 met the threshold for inclusion (≥ 70% of respondents voted critical: 26 SPIRIT-DEFINE, 34 CONSORT-DEFINE), with the remaining 20 items to be further discussed at the consensus meeting. The parallel PPIE work resulted in the development of an EPDF lay summary toolkit consisting of a template with guidance notes and an exemplar.

CONCLUSIONS

By detailing the development journey of the DEFINE study and the decisions undertaken, we envision that this will enhance understanding and help researchers in the development of future guidelines. The SPIRIT-DEFINE and CONSORT-DEFINE guidelines will allow investigators to effectively address essential items that should be present in EPDF trial protocols and reports, thereby promoting transparency, comprehensiveness, and reproducibility.

TRIAL REGISTRATION

SPIRIT-DEFINE and CONSORT-DEFINE are registered with the EQUATOR Network ( https://www.equator-network.org/ ).

Eliminating Ambiguous Treatment Effects Using Estimands

Most reported treatment effects in medical research studies are ambiguously defined, which can lead to misinterpretation of study results. This is because most authors do not attempt to describe what the treatment effect represents, and instead require readers to deduce this based on the reported statistical methods. However, this approach is challenging, because many methods provide counterintuitive results. For example, some methods include data from all patients, yet the resulting treatment effect applies only to a subset of patients, whereas other methods will exclude certain patients while results will apply to everyone. Additionally, some analyses provide estimates pertaining to hypothetical settings in which patients never die or discontinue treatment. Herein we introduce estimands as a solution to the aforementioned problem. An estimand is a clear description of what the treatment effect represents, thus saving readers the necessity of trying to infer this from study methods and potentially getting it wrong. We provide examples of how estimands can remove ambiguity from reported treatment effects and describe their current use in practice. The crux of our argument is that readers should not have to infer what investigators are estimating; they should be told explicitly.

Animals, quality and the pursuit of relevance

In 2021, the National Institutes of Health Advisory Committee to the Director (ACD) announced recommendations to improve the reproducibility of biomedical research using animals. In response, The Jackson Laboratory faculty and institutional leaders identified key strategies to further address this important issue. Taking inspiration from the evolution of clinical trials over recent decades in response to similar challenges, we identified opportunities for improvement, including establishment of common standards, use of genetically diverse populations, requirement for robust study design with appropriate statistical methods, and improvement in public databases to facilitate meta-analyses. In this Perspective, we share our response to ACD recommendations, with a specific focus on mouse models, with the aim of promoting continued active dialogue among researchers, using any animal system, worldwide. Such discussion will help to inform the biomedical community about these recommendations and further support their much-needed implementation.

A systematic review of chiropractic care for fall prevention: rationale, state of the evidence, and recommendations for future research

BACKGROUND

Falls in older adults are a significant and growing public health concern. There are multiple risk factors associated with falls that may be addressed within the scope of chiropractic training and licensure. Few attempts have been made to summarize existing evidence on multimodal chiropractic care and fall risk mitigation. Therefore, the broad purpose of this review was to summarize this research to date. BODY: Systematic review was conducted following PRISMA guidelines. Databases searched included PubMed, Embase, Cochrane Library, PEDro, and Index of Chiropractic Literature. Eligible study designs included randomized controlled trials (RCT), prospective non-randomized controlled, observational, and cross-over studies in which multimodal chiropractic care was the primary intervention and changes in gait, balance and/or falls were outcomes. Risk of bias was also assessed using the 8-item Cochrane Collaboration Tool. The original search yielded 889 articles; 21 met final eligibility including 10 RCTs. One study directly measured the frequency of falls (underpowered secondary outcome) while most studies assessed short-term measurements of gait and balance. The overall methodological quality of identified studies and findings were mixed, limiting interpretation regarding the potential impact of chiropractic care on fall risk to qualitative synthesis.

CONCLUSION

Little high-quality research has been published to inform how multimodal chiropractic care can best address and positively influence fall prevention. We propose strategies for building an evidence base to inform the role of multimodal chiropractic care in fall prevention and outline recommendations for future research to fill current evidence gaps.

Evaluating how clear the questions being investigated in randomised trials are: systematic review of estimands

OBJECTIVES

To evaluate how often the precise research question being addressed about an intervention (the estimand) is stated or can be determined from reported methods, and to identify what types of questions are being investigated in phase 2-4 randomised trials.

DESIGN

Systematic review of the clarity of research questions being investigated in randomised trials in 2020 in six leading general medical journals.

DATA SOURCE

PubMed search in February 2021.

ELIGIBILITY CRITERIA FOR SELECTING STUDIES

Phase 2-4 randomised trials, with no restrictions on medical conditions or interventions. Cluster randomised, crossover, non-inferiority, and equivalence trials were excluded.

MAIN OUTCOME MEASURES

Number of trials that stated the precise primary question being addressed about an intervention (ie, the primary estimand), or for which the primary estimand could be determined unambiguously from the reported methods using statistical knowledge. Strategies used to handle post-randomisation events that affect the interpretation or existence of patient outcomes, such as intervention discontinuations or uses of additional drug treatments (known as intercurrent events), and the corresponding types of questions being investigated.

RESULTS

255 eligible randomised trials were identified. No trials clearly stated all the attributes of the estimand. In 117 (46%) of 255 trials, the primary estimand could be determined from the reported methods. Intercurrent events were reported in 242 (95%) of 255 trials; but the handling of these could only be determined in 125 (49%) of 255 trials. Most trials that provided this information considered the occurrence of intercurrent events as irrelevant in the calculation of the treatment effect and assessed the effect of the intervention regardless (96/125, 77%)-that is, they used a treatment policy strategy. Four (4%) of 99 trials with treatment non-adherence owing to adverse events estimated the treatment effect in a hypothetical setting (ie, the effect as if participants continued treatment despite adverse events), and 19 (79%) of 24 trials where some patients died estimated the treatment effect in a hypothetical setting (ie, the effect as if participants did not die).

CONCLUSIONS

The precise research question being investigated in most trials is unclear, mainly because of a lack of clarity on the approach to handling intercurrent events. Clear reporting of estimands is necessary in trial reports so that all stakeholders, including clinicians, patients and policy makers, can make fully informed decisions about medical interventions.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42021238053.