1
|
Berger VW, Bour LJ, Carter K, Chipman JJ, Everett CC, Heussen N, Hewitt C, Hilgers RD, Luo YA, Renteria J, Ryeznik Y, Sverdlov O, Uschner D. A roadmap to using randomization in clinical trials. BMC Med Res Methodol 2021; 21:168. [PMID: 34399696 PMCID: PMC8366748 DOI: 10.1186/s12874-021-01303-z] [Citation(s) in RCA: 31] [Impact Index Per Article: 10.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/24/2020] [Accepted: 04/14/2021] [Indexed: 12/14/2022] Open
Abstract
BACKGROUND Randomization is the foundation of any clinical trial involving treatment comparison. It helps mitigate selection bias, promotes similarity of treatment groups with respect to important known and unknown confounders, and contributes to the validity of statistical tests. Various restricted randomization procedures with different probabilistic structures and different statistical properties are available. The goal of this paper is to present a systematic roadmap for the choice and application of a restricted randomization procedure in a clinical trial. METHODS We survey available restricted randomization procedures for sequential allocation of subjects in a randomized, comparative, parallel group clinical trial with equal (1:1) allocation. We explore statistical properties of these procedures, including balance/randomness tradeoff, type I error rate and power. We perform head-to-head comparisons of different procedures through simulation under various experimental scenarios, including cases when common model assumptions are violated. We also provide some real-life clinical trial examples to illustrate the thinking process for selecting a randomization procedure for implementation in practice. RESULTS Restricted randomization procedures targeting 1:1 allocation vary in the degree of balance/randomness they induce, and more importantly, they vary in terms of validity and efficiency of statistical inference when common model assumptions are violated (e.g. when outcomes are affected by a linear time trend; measurement error distribution is misspecified; or selection bias is introduced in the experiment). Some procedures are more robust than others. Covariate-adjusted analysis may be essential to ensure validity of the results. Special considerations are required when selecting a randomization procedure for a clinical trial with very small sample size. CONCLUSIONS The choice of randomization design, data analytic technique (parametric or nonparametric), and analysis strategy (randomization-based or population model-based) are all very important considerations. Randomization-based tests are robust and valid alternatives to likelihood-based tests and should be considered more frequently by clinical investigators.
Collapse
Affiliation(s)
| | | | - Kerstine Carter
- Boehringer-Ingelheim Pharmaceuticals Inc, Ridgefield, CT USA
| | - Jonathan J. Chipman
- Population Health Sciences, University of Utah School of Medicine, Salt Lake City UT, USA
- Cancer Biostatistics, University of Utah Huntsman Cancer Institute, Salt Lake City UT, USA
| | | | - Nicole Heussen
- RWTH Aachen University, Aachen, Germany
- Medical School, Sigmund Freud University, Vienna, Austria
| | - Catherine Hewitt
- York Trials Unit, Department of Health Sciences, University of York, York, UK
| | | | | | - Jone Renteria
- Open University of Catalonia (UOC) and the University of Barcelona (UB), Barcelona, Spain
- Department of Human Development and Quantitative Methodology, University of Maryland, College Park, MD USA
| | - Yevgen Ryeznik
- BioPharma Early Biometrics & Statistical Innovations, Data Science & AI, R&D BioPharmaceuticals, AstraZeneca, Gothenburg, Sweden
| | - Oleksandr Sverdlov
- Early Development Analytics, Novartis Pharmaceuticals Corporation, NJ East Hanover, USA
| | - Diane Uschner
- Biostatistics Center & Department of Biostatistics and Bioinformatics, George Washington University, DC Washington, USA
| |
Collapse
|
2
|
Hilgers RD, Manolov M, Heussen N, Rosenberger WF. Design and analysis of stratified clinical trials in the presence of bias. Stat Methods Med Res 2020; 29:1715-1727. [PMID: 31074333 PMCID: PMC7270725 DOI: 10.1177/0962280219846146] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
BACKGROUND Among various design aspects, the choice of randomization procedure have to be agreed on, when planning a clinical trial stratified by center. The aim of the paper is to present a methodological approach to evaluate whether a randomization procedure mitigates the impact of bias on the test decision in clinical trial stratified by center. METHODS We use the weighted t test to analyze the data from a clinical trial stratified by center with a two-arm parallel group design, an intended 1:1 allocation ratio, aiming to prove a superiority hypothesis with a continuous normal endpoint without interim analysis and no adaptation in the randomization process. The derivation is based on the weighted t test under misclassification, i.e. ignoring bias. An additive bias model combing selection bias and time-trend bias is linked to different stratified randomization procedures. RESULTS Various aspects to formulate stratified versions of randomization procedures are discussed. A formula for sample size calculation of the weighted t test is derived and used to specify the tolerated imbalance allowed by some randomization procedures. The distribution of the weighted t test under misclassification is deduced, taking the sequence of patient allocation to treatment, i.e. the randomization sequence into account. An additive bias model combining selection bias and time-trend bias at strata level linked to the applied randomization sequence is proposed. With these before mentioned components, the potential impact of bias on the type one error probability depending on the selected randomization sequence and thus the randomization procedure is formally derived and exemplarily calculated within a numerical evaluation study. CONCLUSION The proposed biasing policy and test distribution are necessary to conduct an evaluation of the comparative performance of (stratified) randomization procedure in multi-center clinical trials with a two-arm parallel group design. It enables the choice of the best practice procedure. The evaluation stimulates the discussion about the level of evidence resulting in those kind of clinical trials.
Collapse
Affiliation(s)
| | - Martin Manolov
- Department of Medical Statistics, RWTH
Aachen University, Aachen, Germany
| | - Nicole Heussen
- Department of Medical Statistics, RWTH
Aachen University, Aachen, Germany
- Department of Biostatistics, Sigmund
Freud University, Vienna, Austria
| | | |
Collapse
|
3
|
Rückbeil MV, Hilgers RD, Heussen N. Randomization in survival studies: An evaluation method that takes into account selection and chronological bias. PLoS One 2019; 14:e0217946. [PMID: 31158260 PMCID: PMC6546249 DOI: 10.1371/journal.pone.0217946] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2019] [Accepted: 05/21/2019] [Indexed: 11/23/2022] Open
Abstract
The random allocation of patients to treatments is a crucial step in the design and conduct of a randomized controlled trial. For this purpose, a variety of randomization procedures is available. In the case of imperfect blinding, the extent to which a randomization procedure forces balanced group sizes throughout the allocation process affects the predictability of allocations. As a result, some randomization procedures perform superior with respect to selection bias, whereas others are less susceptible to chronological bias. The choice of a suitable randomization procedure therefore depends on the expected risk for selection and chronological bias within the particular study in question. To enable a sound comparison of different randomization procedures, we introduce a model for the combined effect of selection and chronological bias in randomized studies with a survival outcome. We present an evaluation method to quantify the influence of bias on the test decision of the log-rank test in a randomized parallel group trial with a survival outcome. The effect of selection and chronological bias and the dependence on the study setting are illustrated in a sensitivity analysis. We conclude with a case study to showcase the application of our model for comparing different randomization procedures in consideration of the expected type I error probability.
Collapse
Affiliation(s)
| | | | - Nicole Heussen
- Department of Medical Statistics, RWTH Aachen University, Aachen, Germany
- Center for Biostatistics and Epidemiology, Sigmund Freud Private University, Vienna, Austria
| |
Collapse
|
4
|
Ryeznik Y, Sverdlov O, Hooker AC. Implementing Optimal Designs for Dose-Response Studies Through Adaptive Randomization for a Small Population Group. AAPS JOURNAL 2018; 20:85. [PMID: 30027336 DOI: 10.1208/s12248-018-0242-5] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/24/2018] [Accepted: 06/18/2018] [Indexed: 11/30/2022]
Abstract
In dose-response studies with censored time-to-event outcomes, D-optimal designs depend on the true model and the amount of censored data. In practice, such designs can be implemented adaptively, by performing dose assignments according to updated knowledge of the dose-response curve at interim analysis. It is also essential that treatment allocation involves randomization-to mitigate various experimental biases and enable valid statistical inference at the end of the trial. In this work, we perform a comparison of several adaptive randomization procedures that can be used for implementing D-optimal designs for dose-response studies with time-to-event outcomes with small to moderate sample sizes. We consider single-stage, two-stage, and multi-stage adaptive designs. We also explore robustness of the designs to experimental (chronological and selection) biases. Simulation studies provide evidence that both the choice of an allocation design and a randomization procedure to implement the target allocation impact the quality of dose-response estimation, especially for small samples. For best performance, a multi-stage adaptive design with small cohort sizes should be implemented using a randomization procedure that closely attains the targeted D-optimal design at each stage. The results of the current work should help clinical investigators select an appropriate randomization procedure for their dose-response study.
Collapse
Affiliation(s)
- Yevgen Ryeznik
- Department of Mathematics, Uppsala University, Room Å14133 Lägerhyddsvägen 1, Hus 1, 6 och 7, 751 06, Uppsala, Sweden. .,Department of Pharmaceutical Biosciences, Uppsala University, Uppsala, Sweden.
| | - Oleksandr Sverdlov
- Early Development Biostatistics, Novartis Institutes for Biomedical Research, East Hannover, New Jersey, USA
| | - Andrew C Hooker
- Department of Pharmaceutical Biosciences, Uppsala University, Uppsala, Sweden
| |
Collapse
|
5
|
Hilgers RD, Bogdan M, Burman CF, Dette H, Karlsson M, König F, Male C, Mentré F, Molenberghs G, Senn S. Lessons learned from IDeAl - 33 recommendations from the IDeAl-net about design and analysis of small population clinical trials. Orphanet J Rare Dis 2018; 13:77. [PMID: 29751809 PMCID: PMC5948846 DOI: 10.1186/s13023-018-0820-8] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2017] [Accepted: 05/01/2018] [Indexed: 01/01/2023] Open
Abstract
BACKGROUND IDeAl (Integrated designs and analysis of small population clinical trials) is an EU funded project developing new statistical design and analysis methodologies for clinical trials in small population groups. Here we provide an overview of IDeAl findings and give recommendations to applied researchers. METHOD The description of the findings is broken down by the nine scientific IDeAl work packages and summarizes results from the project's more than 60 publications to date in peer reviewed journals. In addition, we applied text mining to evaluate the publications and the IDeAl work packages' output in relation to the design and analysis terms derived from in the IRDiRC task force report on small population clinical trials. RESULTS The results are summarized, describing the developments from an applied viewpoint. The main result presented here are 33 practical recommendations drawn from the work, giving researchers a comprehensive guidance to the improved methodology. In particular, the findings will help design and analyse efficient clinical trials in rare diseases with limited number of patients available. We developed a network representation relating the hot topics developed by the IRDiRC task force on small population clinical trials to IDeAl's work as well as relating important methodologies by IDeAl's definition necessary to consider in design and analysis of small-population clinical trials. These network representation establish a new perspective on design and analysis of small-population clinical trials. CONCLUSION IDeAl has provided a huge number of options to refine the statistical methodology for small-population clinical trials from various perspectives. A total of 33 recommendations developed and related to the work packages help the researcher to design small population clinical trial. The route to improvements is displayed in IDeAl-network representing important statistical methodological skills necessary to design and analysis of small-population clinical trials. The methods are ready for use.
Collapse
Affiliation(s)
- Ralf-Dieter Hilgers
- Department of Medical Statistics, RWTH Aachen University, Pauwelsstr. 19, D-52074, Aachen, Germany.
| | - Malgorzata Bogdan
- Department of Medical Statistics, RWTH Aachen University, Pauwelsstr. 19, D-52074, Aachen, Germany
| | - Carl-Fredrik Burman
- Department of Medical Statistics, RWTH Aachen University, Pauwelsstr. 19, D-52074, Aachen, Germany
| | - Holger Dette
- Department of Medical Statistics, RWTH Aachen University, Pauwelsstr. 19, D-52074, Aachen, Germany
| | - Mats Karlsson
- Department of Medical Statistics, RWTH Aachen University, Pauwelsstr. 19, D-52074, Aachen, Germany
| | - Franz König
- Department of Medical Statistics, RWTH Aachen University, Pauwelsstr. 19, D-52074, Aachen, Germany
| | - Christoph Male
- Department of Medical Statistics, RWTH Aachen University, Pauwelsstr. 19, D-52074, Aachen, Germany
| | - France Mentré
- Department of Medical Statistics, RWTH Aachen University, Pauwelsstr. 19, D-52074, Aachen, Germany
| | - Geert Molenberghs
- Department of Medical Statistics, RWTH Aachen University, Pauwelsstr. 19, D-52074, Aachen, Germany
| | - Stephen Senn
- Department of Medical Statistics, RWTH Aachen University, Pauwelsstr. 19, D-52074, Aachen, Germany
| |
Collapse
|
6
|
Uschner D, Hilgers RD, Heussen N. The impact of selection bias in randomized multi-arm parallel group clinical trials. PLoS One 2018; 13:e0192065. [PMID: 29385190 PMCID: PMC5792025 DOI: 10.1371/journal.pone.0192065] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2017] [Accepted: 01/16/2018] [Indexed: 11/18/2022] Open
Abstract
The impact of selection bias on the results of clinical trials has been analyzed extensively for trials of two treatments, yet its impact in multi-arm trials is still unknown. In this paper, we investigate selection bias in multi-arm trials by its impact on the type I error probability. We propose two models for selection bias, so-called biasing policies, that both extend the classic guessing strategy by Blackwell and Hodges. We derive the distribution of the F-test statistic under the misspecified outcome model and provide a formula for the type I error probability under selection bias. We apply the presented approach to quantify the influence of selection bias in multi-arm trials with increasing number of treatment groups using a permuted block design for different assumptions and different biasing strategies. Our results confirm previous findings that smaller block sizes lead to a higher proportion of sequences with inflated type I error probability. Astonishingly, our results also show that the proportion of sequences with inflated type I error probability remains constant when the number of treatment groups is increased. Realizing that the impact of selection bias cannot be completely eliminated, we propose a bias adjusted statistical model and show that the power of the statistical test is only slightly deflated for larger block sizes.
Collapse
Affiliation(s)
- Diane Uschner
- Department of Medical Statistics, RWTH Aachen University, Aachen, Germany
- * E-mail:
| | | | - Nicole Heussen
- Department of Medical Statistics, RWTH Aachen University, Aachen, Germany
- Center of Biostatistics and Epidemiology, Department of Evidence Based Medicine, Sigmund Freund University, Vienna, Austria
| |
Collapse
|
7
|
Hilgers RD, Uschner D, Rosenberger WF, Heussen N. ERDO - a framework to select an appropriate randomization procedure for clinical trials. BMC Med Res Methodol 2017; 17:159. [PMID: 29202708 PMCID: PMC5715815 DOI: 10.1186/s12874-017-0428-z] [Citation(s) in RCA: 23] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2017] [Accepted: 11/15/2017] [Indexed: 11/29/2022] Open
Abstract
BACKGROUND Randomization is considered to be a key feature to protect against bias in randomized clinical trials. Randomization induces comparability with respect to known and unknown covariates, mitigates selection bias, and provides a basis for inference. Although various randomization procedures have been proposed, no single procedure performs uniformly best. In the design phase of a clinical trial, the scientist has to decide which randomization procedure to use, taking into account the practical setting of the trial with respect to the potential of bias. Less emphasis has been placed on this important design decision than on analysis, and less support has been available to guide the scientist in making this decision. METHODS We propose a framework that weights the properties of the randomization procedure with respect to practical needs of the research question to be answered by the clinical trial. In particular, the framework assesses the impact of chronological and selection bias on the probability of a type I error. The framework is applied to a case study with a 2-arm parallel group, single center randomized clinical trial with continuous endpoint, with no-interim analysis, 1:1 allocation and no adaptation in the randomization process. RESULTS In so doing, we derive scientific arguments for the selection of an appropriate randomization procedure and develop a template which is illustrated in parallel by a case study. Possible extensions are discussed. CONCLUSION The proposed ERDO framework guides the investigator through a template for the choice of a randomization procedure, and provides easy to use tools for the assessment. The barriers for the thorough reporting and assessment of randomization procedures could be further reduced in the future when regulators and pharmaceutical companies employ similar, standardized frameworks for the choice of a randomization procedure.
Collapse
Affiliation(s)
- Ralf-Dieter Hilgers
- Department of Medical Statistics, RWTH Aachen University Aachen, Pauwelsstrasse 19, Aachen, Germany
| | - Diane Uschner
- Department of Medical Statistics, RWTH Aachen University Aachen, Pauwelsstrasse 19, Aachen, Germany
| | - William F. Rosenberger
- Department of Statistics, George Mason University, 4400 University Drive, Fairfax, 22030 VA USA
| | - Nicole Heussen
- Department of Medical Statistics, RWTH Aachen University Aachen, Pauwelsstrasse 19, Aachen, Germany
- Center of Biostatistics and Epidemiology, Sigmund Freud University, Freudplatz 1, Vienna, 1020 Austria
| |
Collapse
|
8
|
Abstract
Randomized controlled clinical trials are regarded as the gold standard for comparing different clinical interventions, but generally their conduct is operationally cumbersome, time-consuming, and expensive. Studies and investigations based on clinical routine data on the contrary utilize existing data acquired under real-life conditions and are increasingly popular among practitioners. In this paper, methodological aspects of studies based on clinical routine data are discussed. Important limitations and considerations as well as unique strengths of these types of studies are indicated and exemplarily demonstrated in a recent real-case study based on clinical routine data. In addition two simulation studies reveal the impact of bias in studies based on clinical routine data on the type I error rate and false decision rate in favor of the inferior intervention. It is concluded that correctly analyzing clinical routine data yields a valuable addition to clinical research; however, as a result of a lack of statistical foundation, internal validity, and comparability, generalizing results and inferring properties derived from clinical routine data to all patients of interest has to be considered with extreme caution. FUNDING Grünenthal GmbH.
Collapse
Affiliation(s)
- Lieven Nils Kennes
- Department of Economics and Business Administration, University of Applied Sciences Stralsund, Stralsund, Germany.
| |
Collapse
|
9
|
Rückbeil MV, Hilgers RD, Heussen N. Assessing the impact of selection bias on test decisions in trials with a time-to-event outcome. Stat Med 2017; 36:2656-2668. [PMID: 28417471 PMCID: PMC5516162 DOI: 10.1002/sim.7299] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/28/2016] [Revised: 02/07/2017] [Accepted: 03/11/2017] [Indexed: 11/18/2022]
Abstract
If past treatment assignments are unmasked, selection bias may arise even in randomized controlled trials. The impact of such bias can be measured by considering the type I error probability. In case of a normally distributed outcome, there already exists a model accounting for selection bias that permits calculating the corresponding type I error probabilities. To model selection bias for trials with a time‐to‐event outcome, we introduce a new biasing policy for exponentially distributed data. Using this biasing policy, we derive an exact formula to compute type I error probabilities whenever an F‐test is performed and no observations are censored. Two exemplary settings, with and without random censoring, are considered in order to illustrate how our results can be applied to compare distinct randomization procedures with respect to their performance in the presence of selection bias. © 2017 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.
Collapse
Affiliation(s)
- Marcia Viviane Rückbeil
- Department of Medical Statistics, RWTH Aachen University, Pauwelsstr. 30, 52074, Aachen, Germany
| | - Ralf-Dieter Hilgers
- Department of Medical Statistics, RWTH Aachen University, Pauwelsstr. 30, 52074, Aachen, Germany
| | - Nicole Heussen
- Department of Medical Statistics, RWTH Aachen University, Pauwelsstr. 30, 52074, Aachen, Germany.,Center for Biostatistics and Epidemiology, Medical School, Sigmund Freud Private University, Freudplatz 1, 1020, Vienna, Austria
| |
Collapse
|
10
|
Paludan-Müller A, Teindl Laursen DR, Hróbjartsson A. Mechanisms and direction of allocation bias in randomised clinical trials. BMC Med Res Methodol 2016; 16:133. [PMID: 27717321 PMCID: PMC5055724 DOI: 10.1186/s12874-016-0235-y] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/12/2016] [Accepted: 09/27/2016] [Indexed: 11/12/2022] Open
Abstract
BACKGROUND Selective allocation of patients into the compared groups of a randomised trial may cause allocation bias, but the mechanisms behind the bias and its directionality are incompletely understood. We therefore analysed the mechanisms and directionality of allocation bias in randomised clinical trials. METHODS Two systematic reviews and a theoretical analysis. We conducted one systematic review of empirical studies of motives/methods for deciphering patient allocation sequences; and another review of methods publications commenting on allocation bias. We theoretically analysed the mechanisms of allocation bias and hypothesised which main factors predicts its direction. RESULTS Three empirical studies addressed motives/methods for deciphering allocation sequences. Main motives included ensuring best care for patients and ensuring best outcome for the trial. Main methods included various manipulations with randomisation envelopes. Out of 57 methods publications 11 (19 %) mentioned explicitly that allocation bias can go in either direction. We hypothesised that the direction of allocation bias is mainly decided by the interaction between the patient allocators' motives and treatment preference. CONCLUSION Inadequate allocation concealment may exaggerate treatment effects in some trials while underestimate effects in others. Our hypothesis provides a theoretical overview of the main factors responsible for the direction of allocation bias.
Collapse
Affiliation(s)
| | | | - Asbjørn Hróbjartsson
- The Nordic Cochrane Centre, Rigshospitalet 7811, Copenhagen, Denmark
- Centre for Evidence-Based Medicine, University of Southern Denmark and Odense University Hospital, Odense, Denmark
| |
Collapse
|
11
|
Hilgers RD, Roes K, Stallard N. Directions for new developments on statistical design and analysis of small population group trials. Orphanet J Rare Dis 2016; 11:78. [PMID: 27301273 PMCID: PMC4908723 DOI: 10.1186/s13023-016-0464-5] [Citation(s) in RCA: 27] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2016] [Accepted: 06/03/2016] [Indexed: 11/16/2022] Open
Abstract
Background Most statistical design and analysis methods for clinical trials have been developed and evaluated where at least several hundreds of patients could be recruited. These methods may not be suitable to evaluate therapies if the sample size is unavoidably small, which is usually termed by small populations. The specific sample size cut off, where the standard methods fail, needs to be investigated. In this paper, the authors present their view on new developments for design and analysis of clinical trials in small population groups, where conventional statistical methods may be inappropriate, e.g., because of lack of power or poor adherence to asymptotic approximations due to sample size restrictions. Method Following the EMA/CHMP guideline on clinical trials in small populations, we consider directions for new developments in the area of statistical methodology for design and analysis of small population clinical trials. We relate the findings to the research activities of three projects, Asterix, IDeAl, and InSPiRe, which have received funding since 2013 within the FP7-HEALTH-2013-INNOVATION-1 framework of the EU. As not all aspects of the wide research area of small population clinical trials can be addressed, we focus on areas where we feel advances are needed and feasible. Results The general framework of the EMA/CHMP guideline on small population clinical trials stimulates a number of research areas. These serve as the basis for the three projects, Asterix, IDeAl, and InSPiRe, which use various approaches to develop new statistical methodology for design and analysis of small population clinical trials. Small population clinical trials refer to trials with a limited number of patients. Small populations may result form rare diseases or specific subtypes of more common diseases. New statistical methodology needs to be tailored to these specific situations. Conclusion The main results from the three projects will constitute a useful toolbox for improved design and analysis of small population clinical trials. They address various challenges presented by the EMA/CHMP guideline as well as recent discussions about extrapolation. There is a need for involvement of the patients’ perspective in the planning and conduct of small population clinical trials for a successful therapy evaluation.
Collapse
Affiliation(s)
- Ralf-Dieter Hilgers
- Department of Medical Statistics, RWTH Aachen University, Pauwelsstr 30, D-52074, Aachen, Germany.
| | - Kit Roes
- Quality and Patient Safety, Biostatistics, UMC Utrecht, Utrecht, Netherlands
| | - Nigel Stallard
- Statistics and Epidemiology, Warwick Medical School, University of Warwick, Coventry, UK
| | | |
Collapse
|
12
|
Kahan BC, Rehal S, Cro S. Risk of selection bias in randomised trials. Trials 2015; 16:405. [PMID: 26357929 PMCID: PMC4566301 DOI: 10.1186/s13063-015-0920-x] [Citation(s) in RCA: 80] [Impact Index Per Article: 8.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/24/2015] [Accepted: 08/20/2015] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Selection bias occurs when recruiters selectively enrol patients into the trial based on what the next treatment allocation is likely to be. This can occur even if appropriate allocation concealment is used if recruiters can guess the next treatment assignment with some degree of accuracy. This typically occurs in unblinded trials when restricted randomisation is implemented to force the number of patients in each arm or within each centre to be the same. Several methods to reduce the risk of selection bias have been suggested; however, it is unclear how often these techniques are used in practice. METHODS We performed a review of published trials which were not blinded to assess whether they utilised methods for reducing the risk of selection bias. We assessed the following techniques: (a) blinding of recruiters; (b) use of simple randomisation; (c) avoidance of stratification by site when restricted randomisation is used; (d) avoidance of permuted blocks if stratification by site is used; and (e) incorporation of prognostic covariates into the randomisation procedure when restricted randomisation is used. We included parallel group, individually randomised phase III trials published in four general medical journals (BMJ, Journal of the American Medical Association, The Lancet, and New England Journal of Medicine) in 2010. RESULTS We identified 152 eligible trials. Most trials (98%) provided no information on whether recruiters were blind to previous treatment allocations. Only 3% of trials used simple randomisation; 63% used some form of restricted randomisation, and 35% did not state the method of randomisation. Overall, 44% of trials were stratified by site of recruitment; 27% were not, and 29% did not report this information. Most trials that did stratify by site of recruitment used permuted blocks (58%), and only 15% reported using random block sizes. Many trials that used restricted randomisation also included prognostic covariates in the randomisation procedure (56%). CONCLUSIONS The risk of selection bias could not be ascertained for most trials due to poor reporting. Many trials which did provide details on the randomisation procedure were at risk of selection bias due to a poorly chosen randomisation methods. Techniques to reduce the risk of selection bias should be more widely implemented.
Collapse
Affiliation(s)
- Brennan C Kahan
- Pragmatic Clinical Trials Unit, Queen Mary University of London, E1 2AB, London, UK.
| | - Sunita Rehal
- MRC Clinical Trials Unit at UCL, WC2B 6NH, London, UK.
| | - Suzie Cro
- MRC Clinical Trials Unit at UCL, WC2B 6NH, London, UK.
| |
Collapse
|
13
|
Kahan BC, Morris TP. Reporting and analysis of trials using stratified randomisation in leading medical journals: review and reanalysis. BMJ 2012; 345:e5840. [PMID: 22983531 PMCID: PMC3444136 DOI: 10.1136/bmj.e5840] [Citation(s) in RCA: 201] [Impact Index Per Article: 16.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
OBJECTIVES To assess how often stratified randomisation is used, whether analysis adjusted for all balancing variables, and whether the method of randomisation was adequately reported, and to reanalyse a previously reported trial to assess the impact of ignoring balancing factors in the analysis. DESIGN Review of published trials and reanalysis of a previously reported trial. SETTING Four leading general medical journals (BMJ, Journal of the American Medical Association, Lancet, and New England Journal of Medicine) and the second Multicenter Intrapleural Sepsis Trial (MIST2). PARTICIPANTS 258 trials published in 2010 in the four journals. Cluster randomised, crossover, non-randomised, single arm, and phase I or II trials were excluded, as were trials reporting secondary analyses, interim analyses, or results that had been previously published in 2010. MAIN OUTCOME MEASURES Whether the method of randomisation was adequately reported, how often balanced randomisation was used, and whether balancing factors were adjusted for in the analysis. RESULTS Reanalysis of MIST2 showed that an unadjusted analysis led to larger P values and a loss of power. The review of published trials showed that balanced randomisation was common, with 163 trials (63%) using at least one balancing variable. The most common methods of balancing were stratified permuted blocks (n=85) and minimisation (n=27). The method of randomisation was unclear in 37% of trials. Most trials that balanced on centre or prognostic factors were not adequately analysed; only 26% of trials adjusted for all balancing factors in their primary analysis. Trials that did not adjust for balancing factors in their analysis were less likely to show a statistically significant result (unadjusted 57% v adjusted 78%, P=0.02). CONCLUSION Balancing on centre or prognostic factors is common in trials but often poorly described, and the implications of balancing are poorly understood. Trialists should adjust their primary analysis for balancing factors to obtain correct P values and confidence intervals and to avoid an unnecessary loss in power.
Collapse
Affiliation(s)
- Brennan C Kahan
- MRC Clinical Trials Unit, Aviation House, 125 Kingsway, London WC2B 6NH, UK.
| | | |
Collapse
|