1
|
Andriessen C, Blom MT, van Hoek BACE, de Boer AW, Denig P, de Wit GA, Swart K, de Rooij-Peek A, van Marum RJ, Hugtenburg JG, Slottje P, van Raalte D, van Bloemendaal L, Herings R, Nijpels G, Vos RC, Elders PJM. A deprescribing programme aimed to optimise blood glucose-lowering medication in older people with type 2 diabetes mellitus, the OMED2-study: the study protocol for a randomised controlled trial. Trials 2024; 25:505. [PMID: 39049109 PMCID: PMC11271055 DOI: 10.1186/s13063-024-08249-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2024] [Accepted: 06/14/2024] [Indexed: 07/27/2024] Open
Abstract
BACKGROUND Older patients with type 2 diabetes mellitus (T2D) have an increased risk of hypoglycaemic episodes when using sulphonylureas or insulin. In the Netherlands, guidelines exist for reducing glucose-lowering medication in older patients. However, evidence is lacking that a medication reduction in older patients can be safely pursued. Here, we will examine if promoting the deprescribing of insulin/sulphonylureas with a deprescribing programme (DPP) in general practice affects T2D-complications in older overtreated patients. METHODS We will perform a 1:1 cluster randomised controlled trial in 86 general practices in the Netherlands. The DPP will consist of education sessions with general practitioners and practice nurses about reducing glucose-lowering medication in older patients (≥ 70 years). Topics of the sessions include the necessity of deprescribing, tools to initiate deprescribing and strategies to discuss deprescribing with patients (shared decision making). The DPP further includes a support programme with practice visits. The study will employ a selection tool to identify possibly overtreated older patients from the electronic medical records of the general practitioner. Eligibility for enrolment in the study will be based on HbA1c targets indicated by the Dutch guidelines, which depend on age, diabetes duration, presence of frailty, and life expectancy. The control group will provide usual care. We aim to include 406 patients. The follow-up period will be 2 years. For the primary outcome, the effect of the DPP on T2D-complications will be assessed by counting the cumulative incidence of events related to under- and overtreatment in T2D as registered in the electronic medical records. We shall perform an intention-to-treat analysis and an analysis including only patients for whom deprescribing was initiated. The implementation of the DPP in general practice will be evaluated quantitatively and qualitatively using the Extended Normalisation Process Theory (ENPT) and the Reach, Efficacy - Adoption, Implementation and Maintenance (RE-AIM) model. Other secondary outcomes include quality of life, cognitive functioning, events related to overtreatment or undertreatment, biomarkers of health, amount of blood glucose-lowering medication prescriptions, and cost-effectiveness. DISCUSSION This study will provide insight into the safety and feasibility of a programme aimed at deprescribing sulphonylureas/insulin in older people with T2D who are treated in general practice. TRIAL REGISTRATION ISRCTN Registry, ISRCTN50008265 , registered 09 March, 2023.
Collapse
Affiliation(s)
- Charlotte Andriessen
- Department of General Practice, Amsterdam UMC, Location Vrije Universiteit, Meibergdreef 15, Amsterdam, 1105AZ, the Netherlands
- Amsterdam Public Health Research Institute, Amsterdam, the Netherlands
| | - Marieke T Blom
- Department of General Practice, Amsterdam UMC, Location Vrije Universiteit, Meibergdreef 15, Amsterdam, 1105AZ, the Netherlands
- Amsterdam Public Health Research Institute, Amsterdam, the Netherlands
| | - Beryl A C E van Hoek
- Department of General Practice, Amsterdam UMC, Location Vrije Universiteit, Meibergdreef 15, Amsterdam, 1105AZ, the Netherlands
- Amsterdam Public Health Research Institute, Amsterdam, the Netherlands
| | - Anna W de Boer
- Department of Public Health and Primary Care / Health Campus The Hague, LUMC, Leiden, The Netherlands
| | - Petra Denig
- Department of Clinical Pharmacy and Pharmacology, University of Groningen, University Medical Center Groningen, Groningen, Netherlands
| | - G Ardine de Wit
- Department of Health Sciences, Faculty of Beta Sciences, Vrije Universiteit Amsterdam, Amsterdam, the Netherlands
- Centre for Public Health, Healthcare and Society, National Institute of Public Health and the Environment, Bilthoven, the Netherlands
| | - Karin Swart
- PHARMO Institute for Drug Outcomes Studies, Utrecht, The Netherlands
| | | | - Rob J van Marum
- Department of Geriatric Medicine, Jeroen Bosch Hospital, 's-Hertogenbosch, the Netherlands
- Department of Clinical Pharmacology, Jeroen Bosch Hospital, 's-Hertogenbosch, the Netherlands
- Department of Elderly Care Medicine, Amsterdam University Medical Center, Amsterdam, the Netherlands
| | - Jacqueline G Hugtenburg
- Department of General Practice, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands
| | - Pauline Slottje
- Department of General Practice, Amsterdam UMC, Location Vrije Universiteit, Meibergdreef 15, Amsterdam, 1105AZ, the Netherlands
- Amsterdam Public Health Research Institute, Amsterdam, the Netherlands
| | - Daniël van Raalte
- Diabetes Center, Department of Internal Medicine, Amsterdam University Medical Center, Amsterdam, Netherlands
| | - Liselotte van Bloemendaal
- Department of Internal Medicine - Geriatrics, Amsterdam University Medical Center, Amsterdam, Netherlands
| | - Ron Herings
- PHARMO Institute for Drug Outcomes Studies, Utrecht, The Netherlands
| | - Giel Nijpels
- Department of General Practice, Amsterdam UMC, Location Vrije Universiteit, Meibergdreef 15, Amsterdam, 1105AZ, the Netherlands
- Amsterdam Public Health Research Institute, Amsterdam, the Netherlands
| | - Rimke C Vos
- Department of Public Health and Primary Care / Health Campus The Hague, LUMC, Leiden, The Netherlands
| | - Petra J M Elders
- Department of General Practice, Amsterdam UMC, Location Vrije Universiteit, Meibergdreef 15, Amsterdam, 1105AZ, the Netherlands.
- Amsterdam Public Health Research Institute, Amsterdam, the Netherlands.
| |
Collapse
|
2
|
Sherry AD, Msaouel P, McCaw ZR, Abi Jaoude J, Hsu EJ, Kouzy R, Patel R, Yang Y, Lin TA, Taniguchi CM, Rödel C, Fokas E, Tang C, Fuller CD, Minsky B, Meirson T, Sun R, Ludmir EB. Prevalence and implications of significance testing for baseline covariate imbalance in randomised cancer clinical trials: The Table 1 Fallacy. Eur J Cancer 2023; 194:113357. [PMID: 37827064 DOI: 10.1016/j.ejca.2023.113357] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2023] [Revised: 09/18/2023] [Accepted: 09/20/2023] [Indexed: 10/14/2023]
Abstract
BACKGROUND The 'Table 1 Fallacy' refers to the unsound use of significance testing for comparing the distributions of baseline variables between randomised groups to draw erroneous conclusions about balance or imbalance. We performed a cross-sectional study of the Table 1 Fallacy in phase III oncology trials. METHODS From ClinicalTrials.gov, 1877 randomised trials were screened. Multivariable logistic regressions evaluated predictors of the Table 1 Fallacy. RESULTS A total of 765 randomised controlled trials involving 553,405 patients were analysed. The Table 1 Fallacy was observed in 25% of trials (188 of 765), with 3% of comparisons deemed significant (59 of 2353), approximating the typical 5% type I error assertion probability. Application of trial-level multiplicity corrections reduced the rate of significant findings to 0.3% (six of 2345 tests). Factors associated with lower odds of the Table 1 Fallacy included industry sponsorship (adjusted odds ratio [aOR] 0.29, 95% confidence interval [CI] 0.18-0.47; multiplicity-corrected P < 0.0001), larger trial size (≥795 versus <280 patients; aOR 0.32, 95% CI 0.19-0.53; multiplicity-corrected P = 0.0008), and publication in a European versus American journal (aOR 0.06, 95% CI 0.03-0.13; multiplicity-corrected P < 0.0001). CONCLUSIONS This study highlights the persistence of the Table 1 Fallacy in contemporary oncology randomised controlled trials, with one of every four trials testing for baseline differences after randomisation. Significance testing is a suboptimal method for identifying unsound randomisation procedures and may encourage misleading inferences. Journal-level enforcement is a possible strategy to help mitigate this fallacy.
Collapse
Affiliation(s)
- Alexander D Sherry
- Department of Radiation Oncology, Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Pavlos Msaouel
- Department of Genitourinary Medical Oncology, Division of Cancer Medicine, The University of Texas MD Anderson Cancer Center, Houston, TX, USA; Department of Translational Molecular Pathology, Division of Pathology/Lab Medicine, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Zachary R McCaw
- Insitro, South San Francisco, CA, USA; Department of Biostatistics, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
| | - Joseph Abi Jaoude
- Department of Radiation Oncology, Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA; Department of Radiation Oncology, Stanford University, Stanford, CA, USA
| | - Eric J Hsu
- Department of Radiation Oncology, University of Texas Southwestern Medical Center, Dallas, TX, USA
| | - Ramez Kouzy
- Department of Radiation Oncology, Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Roshal Patel
- Department of Radiation Oncology, Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA; Department of Radiation Oncology, Memorial Sloan-Kettering Cancer Center, New York, NY, USA
| | - Yumeng Yang
- Department of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, TX, USA
| | - Timothy A Lin
- Department of Radiation Oncology and Molecular Radiation Sciences, Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Cullen M Taniguchi
- Department of Gastrointestinal Radiation Oncology, Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA; Department of Experimental Radiation Oncology, Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Claus Rödel
- Department of Radiotherapy and Oncology, University of Frankfurt, Frankfurt, Germany; Frankfurt Cancer Institute, Frankfurt, Germany; German Cancer Research Center (DKFZ), Heidelberg, German Cancer Consortium (DKTK), Partner Site Frankfurt am Main, Frankfurt, Germany
| | - Emmanouil Fokas
- Department of Radiotherapy and Oncology, University of Frankfurt, Frankfurt, Germany; Frankfurt Cancer Institute, Frankfurt, Germany; German Cancer Research Center (DKFZ), Heidelberg, German Cancer Consortium (DKTK), Partner Site Frankfurt am Main, Frankfurt, Germany
| | - Chad Tang
- Department of Translational Molecular Pathology, Division of Pathology/Lab Medicine, The University of Texas MD Anderson Cancer Center, Houston, TX, USA; Department of Genitourinary Radiation Oncology, Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Clifton David Fuller
- Department of Radiation Oncology, Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Bruce Minsky
- Department of Gastrointestinal Radiation Oncology, Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Tomer Meirson
- Davidoff Cancer Center, Rabin Medical Center, Petach Tikva, Israel
| | - Ryan Sun
- Department of Biostatistics, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Ethan B Ludmir
- Department of Gastrointestinal Radiation Oncology, Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA; Department of Biostatistics, The University of Texas MD Anderson Cancer Center, Houston, TX, USA.
| |
Collapse
|
3
|
Azzolina D, Comoretto R, Da Dalt L, Bressan S, Gregori D. A SuperLearner-enforced approach for the estimation of treatment effect in pediatric trials. Digit Health 2023; 9:20552076231191967. [PMID: 37559827 PMCID: PMC10408313 DOI: 10.1177/20552076231191967] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2023] [Accepted: 07/18/2023] [Indexed: 08/11/2023] Open
Abstract
Background Randomized Clinical Trials (RCT) represent the gold standard among scientific evidence. RCTs are tailored to control selection bias and the confounding effect of baseline characteristics on the effect of treatment. However, trial conduction and enrolment procedures could be challenging, especially for rare diseases and paediatric research. In these research frameworks, the treatment effect estimation could be compromised. A potential countermeasure is to develop predictive models on the probability of the baseline disease based on previously collected observational data. Machine learning (ML) algorithms have recently become attractive in clinical research because of their flexibility and improved performance compared to standard statistical methods in developing predictive models. Objective This manuscript proposes an ML-enforced treatment effect estimation procedure based on an ensemble SuperLearner (SL) approach, trained on historical observational data, to control the confounding effect. Methods The REnal SCarring Urinary infEction trial served as a motivating example. Historical observational study data have been simulated through 10,000 Monte Carlo (MC) runs. Hypothetical RCTs have been also simulated, for each MC run, assuming different treatment effects of antibiotics combined with steroids. For each MC simulation, the SL tool has been applied to the simulated observational data. Furthermore, the average treatment effect (ATE), has been estimated on the trial data and adjusted for the SL predicted probability of renal scar. Results The simulation results revealed an increased power in ATE estimation for the SL-enforced estimation compared to the unadjusted estimates for all the algorithms composing the ensemble SL.
Collapse
Affiliation(s)
- Danila Azzolina
- Department of Environmental and Preventive Science, University of Ferrara, Ferrara, Italy
| | - Rosanna Comoretto
- Department of Public Health and Pediatrics, University of Turin, Turin, Italy
| | - Liviana Da Dalt
- Department of Women's and Children's Health, University of Padova, Padova, Italy
| | - Silvia Bressan
- Department of Women's and Children's Health, University of Padova, Padova, Italy
| | - Dario Gregori
- Unit of Biostatistics, Epidemiology and Public Health, Department of Cardiac, Thoracic, Vascular Sciences and Public Health, University of Padova, Padova, Italy
| |
Collapse
|
4
|
Risk of bias: why measure it, and how? Eye (Lond) 2022; 36:346-348. [PMID: 34594009 PMCID: PMC8807607 DOI: 10.1038/s41433-021-01759-9] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/09/2021] [Revised: 08/12/2021] [Accepted: 08/17/2021] [Indexed: 02/03/2023] Open
|
5
|
Clark L, Burke L, Margaret Carr R, Coleman E, Roberts G, Torgerson DJ. A review found small variable blocking schemes may not protect against selection bias in randomized controlled trials. J Clin Epidemiol 2021; 141:90-98. [PMID: 34520850 DOI: 10.1016/j.jclinepi.2021.09.009] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/10/2021] [Revised: 08/06/2021] [Accepted: 09/07/2021] [Indexed: 10/20/2022]
Abstract
OBJECTIVE Blocking is associated with prediction of the allocation sequence and subversion. This paper explores if blocking strategies lead to an increase in baseline age heterogeneity (a marker for potential subversion) and, whether the use of blocking is changing over time. STUDY DESIGN AND SETTINGS The British Medical Journal, Journal of the American Medical Association, The Lancet and the New England Journal of Medicine were hand searched to identify open RCTs published in January between 2001 and 2020. To explore heterogeneity of baseline age meta-analyses were performed on trials implementing blocking, minimization, and simple randomization. RESULTS One hundred seventy-nine open RCTs were identified: nine (5.0%) undertook simple randomization, 104 (58.1%) blocking, 25 (13.9%) minimization, and one (0.6%) both. Baseline age heterogeneity of 24% (P= 0.02) was observed in all trials implementing blocking, 62% (P = 0.001) in trials implementing a fixed block of four, 40% (P = 0.07) implementing variable blocks including a 2 and 0% for both simple randomization and minimization. Small block sizes are implemented in modern trials. CONCLUSION Variable block sizes including two are associated with subversion and should not be implemented. If center only stratification is necessary, it should be used alongside larger blocking schemes. Authors should consider alternative methods to restrict randomization.
Collapse
Affiliation(s)
- Laura Clark
- York Trials Unit, Department of Health Sciences, University of York, York, YO10 5DD, United Kingdom.
| | - Lauren Burke
- York Trials Unit, Department of Health Sciences, University of York, York, YO10 5DD, United Kingdom
| | - Rachel Margaret Carr
- York Trials Unit, Department of Health Sciences, University of York, York, YO10 5DD, United Kingdom
| | - Elizabeth Coleman
- York Trials Unit, Department of Health Sciences, University of York, York, YO10 5DD, United Kingdom
| | - Gareth Roberts
- York Trials Unit, Department of Health Sciences, University of York, York, YO10 5DD, United Kingdom
| | - David J Torgerson
- York Trials Unit, Department of Health Sciences, University of York, York, YO10 5DD, United Kingdom
| |
Collapse
|
6
|
Rückbeil MV, Manolov M, Hilgers RD. The Choice of a Randomization Procedure in Survival Studies with Nonproportional Hazards. Stat Biopharm Res 2021. [DOI: 10.1080/19466315.2021.1952894] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
Affiliation(s)
| | - Martin Manolov
- Institute for Computational Genomics, RWTH Aachen University, Aachen, Germany
| | | |
Collapse
|
7
|
Uschner D. Randomization-based inference in the presence of selection bias. Stat Med 2021; 40:2212-2229. [PMID: 33561882 DOI: 10.1002/sim.8898] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2020] [Revised: 12/24/2020] [Accepted: 01/16/2021] [Indexed: 11/05/2022]
Abstract
For the analysis of clinical trials, the study participants are usually assumed to be representative sample of a target population. This assumption is rarely fulfilled in clinical trials, and particularly not if the sample size is small. In addition, covariate imbalances may affect the trial. Randomization tests provide a nonparametric analysis method of the treatment effect that does not rely on population-based assumptions. We propose a nonparametric statistical model that yields a formal basis for randomization tests. We adapt the model for the presence of covariate imbalance in the form of selection bias and investigate the effects of bias on the rejection probability of the randomization test using Monte Carlo simulations. Finally, we show that ancillary statistics can be used to control for the influence of bias. We show that covariate imbalance leads to an inflation of the type I error probability. The proposed nonparametric model allows for the use of ancillary statistics that yield an unbiased adjusted randomization test.
Collapse
Affiliation(s)
- Diane Uschner
- The Biostatistics Center, George Washington University, Rockville, Maryland, USA
| |
Collapse
|
8
|
DEPOSEIN: Take with a grain of salt. Neuro Oncol 2020; 22:1708-1709. [DOI: 10.1093/neuonc/noaa181] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/14/2022] Open
|
9
|
Ter Veer E, van Oijen MGH, van Laarhoven HWM. The Use of (Network) Meta-Analysis in Clinical Oncology. Front Oncol 2019; 9:822. [PMID: 31508373 PMCID: PMC6718703 DOI: 10.3389/fonc.2019.00822] [Citation(s) in RCA: 31] [Impact Index Per Article: 6.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/09/2019] [Accepted: 08/12/2019] [Indexed: 12/13/2022] Open
Abstract
Meta-analysis is important in oncological research to provide a more reliable answer to a clinical research question that was assessed in multiple studies but with inconsistent results. Pair-wise meta-analysis can be applied when comparing two treatments at once, whereas it is possible to compare multiple treatments at once with network meta-analysis (NMA). After careful systematic review of the literature and quality assessment of the identified studies, there are several assumptions in the use of meta-analysis. First, the added value of meta-analysis should be evaluated by examining the comparability of study populations. Second, the appropriate comparator in meta-analysis should be chosen according to the types of comparisons made in individual studies: (1) Experimental and comparator arms are different treatments (A vs. B); (2) Substitution of a conventional treatment by an experimental treatment (A+B vs. A+C); or (3) Addition of an experimental treatment (A+B vs. B). Ideally there is one common comparator treatment, but when there are multiple common comparators, the most efficacious comparator is preferable. Third, treatments can only be adequately pooled in meta-analysis or merged into one treatment node in NMA when considering likewise mechanism of action and similar setting in which treatment is indicated. Fourth, for both pair-wise meta-analysis and NMA, adequate assessment of heterogeneity should be performed and sub-analysis and sensitivity analysis can be applied to objectify a possible confounding factor. Network inconsistency, as statistical manifestation of violating the transitivity assumption, can best be evaluated by node-split modeling. NMA has advantages over pair-wise meta-analysis, such as clarification of inconsistent outcomes from multiple studies including multiple common comparators and indirect effect calculation of missing direct comparisons between important treatments. Also, NMA can provide increased statistical power and cross-validation of the observed treatment effect of weak connections with reasonable network connectivity and sufficient sample-sizes. However, inappropriate use of NMA can cause misleading results, and may emerge when there is low network connectivity, and therefore low statistical power. Furthermore, indirect evidence is still observational and should be interpreted with caution. NMA should therefore preferably be conducted and interpreted by both expert clinicians in the field and an experienced statistician. Finally, the use of meta-analysis can be extended to other areas, for example the identification of prognostic and predictive factors. Also, the integration of evidence from both meta-analysis and expert opinion can improve the construction of prognostic models in real-world databases.
Collapse
Affiliation(s)
- Emil Ter Veer
- Department of Medical Oncology, Cancer Centre Amsterdam, Amsterdam UMC, University of Amsterdam, Amsterdam, Netherlands
| | - Martijn G H van Oijen
- Department of Medical Oncology, Cancer Centre Amsterdam, Amsterdam UMC, University of Amsterdam, Amsterdam, Netherlands
| | - Hanneke W M van Laarhoven
- Department of Medical Oncology, Cancer Centre Amsterdam, Amsterdam UMC, University of Amsterdam, Amsterdam, Netherlands
| |
Collapse
|
10
|
Rückbeil MV, Hilgers RD, Heussen N. Randomization in survival studies: An evaluation method that takes into account selection and chronological bias. PLoS One 2019; 14:e0217946. [PMID: 31158260 PMCID: PMC6546249 DOI: 10.1371/journal.pone.0217946] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2019] [Accepted: 05/21/2019] [Indexed: 11/23/2022] Open
Abstract
The random allocation of patients to treatments is a crucial step in the design and conduct of a randomized controlled trial. For this purpose, a variety of randomization procedures is available. In the case of imperfect blinding, the extent to which a randomization procedure forces balanced group sizes throughout the allocation process affects the predictability of allocations. As a result, some randomization procedures perform superior with respect to selection bias, whereas others are less susceptible to chronological bias. The choice of a suitable randomization procedure therefore depends on the expected risk for selection and chronological bias within the particular study in question. To enable a sound comparison of different randomization procedures, we introduce a model for the combined effect of selection and chronological bias in randomized studies with a survival outcome. We present an evaluation method to quantify the influence of bias on the test decision of the log-rank test in a randomized parallel group trial with a survival outcome. The effect of selection and chronological bias and the dependence on the study setting are illustrated in a sensitivity analysis. We conclude with a case study to showcase the application of our model for comparing different randomization procedures in consideration of the expected type I error probability.
Collapse
Affiliation(s)
| | | | - Nicole Heussen
- Department of Medical Statistics, RWTH Aachen University, Aachen, Germany
- Center for Biostatistics and Epidemiology, Sigmund Freud Private University, Vienna, Austria
| |
Collapse
|
11
|
Saltaji H, Armijo-Olivo S, Cummings GG, Amin M, da Costa BR, Flores-Mir C. Influence of blinding on treatment effect size estimate in randomized controlled trials of oral health interventions. BMC Med Res Methodol 2018; 18:42. [PMID: 29776394 PMCID: PMC5960173 DOI: 10.1186/s12874-018-0491-0] [Citation(s) in RCA: 50] [Impact Index Per Article: 8.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/07/2017] [Accepted: 04/15/2018] [Indexed: 01/18/2023] Open
Abstract
Background Recent methodologic evidence suggests that lack of blinding in randomized trials can result in under- or overestimation of the treatment effect size. The objective of this study is to quantify the extent of bias associated with blinding in randomized controlled trials of oral health interventions. Methods We selected all oral health meta-analyses that included a minimum of five randomized controlled trials. We extracted data, in duplicate, related to nine blinding-related criteria, namely: patient blinding, assessor blinding, care-provider blinding, investigator blinding, statistician blinding, blinding of both patients and assessors, study described as “double blind”, blinding of patients, assessors, and care providers concurrently, and the appropriateness of blinding. We quantified the impact of bias associated with blinding on the magnitude of effect size using a two-level meta-meta-analytic approach with a random effects model to allow for intra- and inter-meta-analysis heterogeneity. Results We identified 540 randomized controlled trials, included in 64 meta-analyses, analyzing data from 137,957 patients. We identified significantly larger treatment effect size estimates in trials that had inadequate patient blinding (difference in treatment effect size = 0.12; 95% CI: 0.00 to 0.23), lack of blinding of both patients and assessors (difference = 0.19; 95% CI: 0.06 to 0.32), and lack of blinding of patients, assessors, and care-providers concurrently (difference = 0.14; 95% CI: 0.03 to 0.25). In contrast, assessor blinding (difference = 0.06; 95% CI: -0.06 to 0.18), caregiver blinding (difference = 0.02; 95% CI: -0.04 to 0.09), principal-investigator blinding (difference = − 0.02; 95% CI: -0.10 to 0.06), describing a trial as “double-blind” (difference = 0.09; 95% CI: -0.05 to 0.22), and lack of an appropriate method of blinding (difference = 0.06; 95% CI: -0.06 to 0.18) were not associated with over- or underestimated treatment effect size. Conclusions We found significant differences in treatment effect size estimates between oral health trials based on lack of patient and assessor blinding. Treatment effect size estimates were 0.19 and 0.14 larger in trials with lack of blinding of both patients and assessors and blinding of patients, assessors, and care-providers concurrently. No significant differences were identified in other blinding criteria. Investigators of oral health systematic reviews should perform sensitivity analyses based on the adequacy of blinding in included trials. Electronic supplementary material The online version of this article (10.1186/s12874-018-0491-0) contains supplementary material, which is available to authorized users.
Collapse
Affiliation(s)
- Humam Saltaji
- Orthodontic Graduate Program, School of Dentistry, Edmonton Clinic Health Academy, University of Alberta, 11405-87 Ave, Edmonton, AB, T6G 1C9, Canada.
| | - Susan Armijo-Olivo
- Faculty of Rehabilitation Medicine, University of Alberta, Edmonton, AB, Canada
| | | | - Maryam Amin
- Division of Pediatric Dentistry, School of Dentistry, University of Alberta, Edmonton, AB, Canada
| | - Bruno R da Costa
- Department of Physical Therapy, Institute of Primary Health Care (BIHAM), Florida International University, Miami, USA.,University of Bern, Bern, Switzerland
| | - Carlos Flores-Mir
- Division of Orthodontics, School of Dentistry, University of Alberta, Edmonton, Canada
| |
Collapse
|
12
|
Uschner D, Hilgers RD, Heussen N. The impact of selection bias in randomized multi-arm parallel group clinical trials. PLoS One 2018; 13:e0192065. [PMID: 29385190 PMCID: PMC5792025 DOI: 10.1371/journal.pone.0192065] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2017] [Accepted: 01/16/2018] [Indexed: 11/18/2022] Open
Abstract
The impact of selection bias on the results of clinical trials has been analyzed extensively for trials of two treatments, yet its impact in multi-arm trials is still unknown. In this paper, we investigate selection bias in multi-arm trials by its impact on the type I error probability. We propose two models for selection bias, so-called biasing policies, that both extend the classic guessing strategy by Blackwell and Hodges. We derive the distribution of the F-test statistic under the misspecified outcome model and provide a formula for the type I error probability under selection bias. We apply the presented approach to quantify the influence of selection bias in multi-arm trials with increasing number of treatment groups using a permuted block design for different assumptions and different biasing strategies. Our results confirm previous findings that smaller block sizes lead to a higher proportion of sequences with inflated type I error probability. Astonishingly, our results also show that the proportion of sequences with inflated type I error probability remains constant when the number of treatment groups is increased. Realizing that the impact of selection bias cannot be completely eliminated, we propose a bias adjusted statistical model and show that the power of the statistical test is only slightly deflated for larger block sizes.
Collapse
Affiliation(s)
- Diane Uschner
- Department of Medical Statistics, RWTH Aachen University, Aachen, Germany
- * E-mail:
| | | | - Nicole Heussen
- Department of Medical Statistics, RWTH Aachen University, Aachen, Germany
- Center of Biostatistics and Epidemiology, Department of Evidence Based Medicine, Sigmund Freund University, Vienna, Austria
| |
Collapse
|
13
|
Yaseen K, Hendrick P, Ismail A, Felemban M, Alshehri MA. The effectiveness of manual therapy in treating cervicogenic dizziness: a systematic review. J Phys Ther Sci 2018; 30:96-102. [PMID: 29410575 PMCID: PMC5788784 DOI: 10.1589/jpts.30.96] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2017] [Accepted: 10/12/2017] [Indexed: 11/24/2022] Open
Abstract
[Purpose] This review provides an evaluation of the evidence for the effectiveness of
using manual therapy to treat cervicogenic dizziness. [Subjects and Methods] The
literature was systematically searched on the May 2, 2016 using the following online
databases: Medline, EMBASE, CINAHL and PEDro. This review included randomised controlled
trials and compared the efficacy of manual therapy for the treatment of cervicogenic
dizziness, compared to other types of intervention. This study measured changes based on
dizziness intensity and frequency. [Results] The primary search found 30 articles, but
only four articles met the inclusion criteria. Assessment of methodological quality was
performed by two researchers using the PEDro scale. The level of evidence was determined
using a recognised grading scale. Three out of the four articles were deemed to have high
methodological quality, while the fourth was rated as moderate quality. The attributed
level of evidence was moderate (level 2). [Conclusion] Manual therapy is potentially
effective for managing cervicogenic dizziness. However, due to the heterogeneity of the
results and techniques and the low number of studies, further research is recommended to
provide conclusive evidence.
Collapse
Affiliation(s)
- Khalid Yaseen
- Department of Physical Therapy, Faculty of Physical Therapy, King Abdul Aziz University: Jeddah, Saudi Arabia
| | - Paul Hendrick
- Physiotherapy Division, School of Health Sciences, University of Nottingham, UK
| | - Ayah Ismail
- Department of Physical Therapy, Faculty of Physical Therapy, King Abdul Aziz University: Jeddah, Saudi Arabia
| | - Mohannad Felemban
- Department of Physical Therapy, Faculty of Physical Therapy, King Abdul Aziz University: Jeddah, Saudi Arabia
| | | |
Collapse
|
14
|
Ismail A, Moore C, Alshishani N, Yaseen K, Alshehri MA. Cognitive behavioural therapy and pain coping skills training for osteoarthritis knee pain management: a systematic review. J Phys Ther Sci 2017; 29:2228-2235. [PMID: 29643612 PMCID: PMC5890238 DOI: 10.1589/jpts.29.2228] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2017] [Accepted: 09/29/2017] [Indexed: 12/17/2022] Open
Abstract
[Purpose] To investigate the effectiveness of cognitive-behavioural therapy (CBT) and
pain coping skills training (PCST) on pain level in adults with osteoarthritis of the knee
(KOA) in comparison with usual care. [Subjects and Methods] Five databases were
systematically searched for relevant randomised controlled trials (RCTs) according to the
selected eligibility criteria (inception to June 7, 2016). PEDro scale was used to assess
the validity of included studies. [Results] Four studies met the inclusion criteria and
all studies had high methodological quality. The total number of participants was 665
across the four included studies. All participants had been diagnosed with KOA (clinical
and radiographic evaluation). The majority of the population were female (71.13%). The
interventions utilised were: CBT, CBT for insomnia (CBT-I), PCST, and PCST combined with
behavioral weight management (BWM) in one study and with exercise in another study. The
outcome measure for pain in KOA was the WOMAC scale. Overall, three studies have reported
clinical improvement in KOA pain perception after 12-month follow-up time point.
[Conclusion] This review has recognised the need for future studies that have rigorous
methodological quality, and investigate the effect of mutual CBT and PCST protocols on KOA
pain and pain-related functional and psychological abilities.
Collapse
Affiliation(s)
- Ayah Ismail
- Department of Physical Therapy, Faculty of Applied Medical Sciences, King Abdul Aziz University: Jeddah, Saudi Arabia
| | - Catherine Moore
- Department of Physical Therapy, Faculty of Applied Medical Sciences, King Abdul Aziz University: Jeddah, Saudi Arabia
| | - Natasha Alshishani
- Department of Physical Therapy, Faculty of Applied Medical Sciences, King Abdul Aziz University: Jeddah, Saudi Arabia
| | - Khalid Yaseen
- Department of Physical Therapy, Faculty of Applied Medical Sciences, King Abdul Aziz University: Jeddah, Saudi Arabia
| | - Mansour Abdullah Alshehri
- Department of Physical Therapy, Faculty of Applied Medical Sciences, King Abdul Aziz University: Jeddah, Saudi Arabia
| |
Collapse
|
15
|
Abstract
Randomized controlled clinical trials are regarded as the gold standard for comparing different clinical interventions, but generally their conduct is operationally cumbersome, time-consuming, and expensive. Studies and investigations based on clinical routine data on the contrary utilize existing data acquired under real-life conditions and are increasingly popular among practitioners. In this paper, methodological aspects of studies based on clinical routine data are discussed. Important limitations and considerations as well as unique strengths of these types of studies are indicated and exemplarily demonstrated in a recent real-case study based on clinical routine data. In addition two simulation studies reveal the impact of bias in studies based on clinical routine data on the type I error rate and false decision rate in favor of the inferior intervention. It is concluded that correctly analyzing clinical routine data yields a valuable addition to clinical research; however, as a result of a lack of statistical foundation, internal validity, and comparability, generalizing results and inferring properties derived from clinical routine data to all patients of interest has to be considered with extreme caution. FUNDING Grünenthal GmbH.
Collapse
Affiliation(s)
- Lieven Nils Kennes
- Department of Economics and Business Administration, University of Applied Sciences Stralsund, Stralsund, Germany.
| |
Collapse
|
16
|
Rückbeil MV, Hilgers RD, Heussen N. Assessing the impact of selection bias on test decisions in trials with a time-to-event outcome. Stat Med 2017; 36:2656-2668. [PMID: 28417471 PMCID: PMC5516162 DOI: 10.1002/sim.7299] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/28/2016] [Revised: 02/07/2017] [Accepted: 03/11/2017] [Indexed: 11/18/2022]
Abstract
If past treatment assignments are unmasked, selection bias may arise even in randomized controlled trials. The impact of such bias can be measured by considering the type I error probability. In case of a normally distributed outcome, there already exists a model accounting for selection bias that permits calculating the corresponding type I error probabilities. To model selection bias for trials with a time‐to‐event outcome, we introduce a new biasing policy for exponentially distributed data. Using this biasing policy, we derive an exact formula to compute type I error probabilities whenever an F‐test is performed and no observations are censored. Two exemplary settings, with and without random censoring, are considered in order to illustrate how our results can be applied to compare distinct randomization procedures with respect to their performance in the presence of selection bias. © 2017 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.
Collapse
Affiliation(s)
- Marcia Viviane Rückbeil
- Department of Medical Statistics, RWTH Aachen University, Pauwelsstr. 30, 52074, Aachen, Germany
| | - Ralf-Dieter Hilgers
- Department of Medical Statistics, RWTH Aachen University, Pauwelsstr. 30, 52074, Aachen, Germany
| | - Nicole Heussen
- Department of Medical Statistics, RWTH Aachen University, Pauwelsstr. 30, 52074, Aachen, Germany.,Center for Biostatistics and Epidemiology, Medical School, Sigmund Freud Private University, Freudplatz 1, 1020, Vienna, Austria
| |
Collapse
|
17
|
Paludan-Müller A, Teindl Laursen DR, Hróbjartsson A. Mechanisms and direction of allocation bias in randomised clinical trials. BMC Med Res Methodol 2016; 16:133. [PMID: 27717321 PMCID: PMC5055724 DOI: 10.1186/s12874-016-0235-y] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/12/2016] [Accepted: 09/27/2016] [Indexed: 11/12/2022] Open
Abstract
BACKGROUND Selective allocation of patients into the compared groups of a randomised trial may cause allocation bias, but the mechanisms behind the bias and its directionality are incompletely understood. We therefore analysed the mechanisms and directionality of allocation bias in randomised clinical trials. METHODS Two systematic reviews and a theoretical analysis. We conducted one systematic review of empirical studies of motives/methods for deciphering patient allocation sequences; and another review of methods publications commenting on allocation bias. We theoretically analysed the mechanisms of allocation bias and hypothesised which main factors predicts its direction. RESULTS Three empirical studies addressed motives/methods for deciphering allocation sequences. Main motives included ensuring best care for patients and ensuring best outcome for the trial. Main methods included various manipulations with randomisation envelopes. Out of 57 methods publications 11 (19 %) mentioned explicitly that allocation bias can go in either direction. We hypothesised that the direction of allocation bias is mainly decided by the interaction between the patient allocators' motives and treatment preference. CONCLUSION Inadequate allocation concealment may exaggerate treatment effects in some trials while underestimate effects in others. Our hypothesis provides a theoretical overview of the main factors responsible for the direction of allocation bias.
Collapse
Affiliation(s)
| | | | - Asbjørn Hróbjartsson
- The Nordic Cochrane Centre, Rigshospitalet 7811, Copenhagen, Denmark
- Centre for Evidence-Based Medicine, University of Southern Denmark and Odense University Hospital, Odense, Denmark
| |
Collapse
|
18
|
Berntsen CF, Kristiansen A, Akl EA, Sandset PM, Jacobsen EM, Guyatt G, Vandvik PO. Compression Stockings for Preventing the Postthrombotic Syndrome in Patients with Deep Vein Thrombosis. Am J Med 2016; 129:447.e1-447.e20. [PMID: 26747198 DOI: 10.1016/j.amjmed.2015.11.031] [Citation(s) in RCA: 20] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/13/2015] [Revised: 11/29/2015] [Accepted: 11/30/2015] [Indexed: 11/16/2022]
Abstract
OBJECTIVE We conducted a systematic review and meta-analysis to address benefits and harms of using elastic compression stockings after lower-extremity deep vein thrombosis. METHODS We searched 7 electronic databases through January 15, 2015, including randomized controlled trials (RCTs)/quasi-randomized trials reporting on elastic compression stocking efficacy on postthrombotic syndrome incidence, recurrent venous thromboembolism, mortality, and acute pain after deep vein thrombosis. Two reviewers independently screened records, extracted data, assessed risk of bias, and assessed confidence in effect estimates using Grading of Recommendations Assessment, Development, and Evaluation methodology. We applied random-effects meta-analysis models. RESULTS We included 5 RCTs (n = 1418) reporting on postthrombotic syndrome. The hazard ratio (HR) for postthrombotic syndrome with elastic compression stockings was 0.69 (95% confidence interval [CI], 0.47-1.02). We have very low confidence in this estimate due to heterogeneity and inclusion of unblinded studies at high risk of bias. Excluding high risk of bias studies, a single large RCT at low risk of bias provided moderate-quality evidence of no effect on postthrombotic syndrome (HR 1.00; 95% CI, 0.81-1.24). Moderate-quality evidence including all 5 studies suggests no effect of elastic compression stockings on recurrent venous thromboembolism (relative risk [RR] 0.88; 95% CI, 0.63-1.24) or mortality (RR 1.00; 95% CI, 0.73-1.37, 5 studies). Moderate-quality evidence from one large RCT does not suggest effect on acute pain after deep vein thrombosis. CONCLUSIONS The highest-quality evidence available suggests no effect of elastic compression stockings on postthrombotic syndrome or pain relief, from a single large RCT. However, results for preventing postthrombotic syndrome differ substantially across studies, and future guideline updates should reflect uncertainty about treatment effects. Elastic compression stockings are unlikely to prevent death or recurrent venous thromboembolism.
Collapse
Affiliation(s)
- Christopher Friis Berntsen
- Department of Internal Medicine, Sykehuset Innlandet Hospital Trust, Gjøvik, Norway; Institute of Health and Society, Faculty of Medicine, University of Oslo, Norway.
| | - Annette Kristiansen
- Department of Internal Medicine, Sykehuset Innlandet Hospital Trust, Gjøvik, Norway; Institute of Health and Society, Faculty of Medicine, University of Oslo, Norway
| | - Elie A Akl
- Department of Internal Medicine, American University of Beirut, Lebanon
| | - Per Morten Sandset
- Department of Haematology, Oslo University Hospital, Norway; Institute of Clinical Medicine, Faculty of Medicine, University of Oslo, Norway
| | - Eva-Marie Jacobsen
- Department of Haematology, Oslo University Hospital, Norway; Institute of Clinical Medicine, Faculty of Medicine, University of Oslo, Norway
| | - Gordon Guyatt
- Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada
| | - Per Olav Vandvik
- Department of Internal Medicine, Sykehuset Innlandet Hospital Trust, Gjøvik, Norway; Institute of Health and Society, Faculty of Medicine, University of Oslo, Norway
| |
Collapse
|
19
|
Kahan BC, Rehal S, Cro S. Risk of selection bias in randomised trials. Trials 2015; 16:405. [PMID: 26357929 PMCID: PMC4566301 DOI: 10.1186/s13063-015-0920-x] [Citation(s) in RCA: 80] [Impact Index Per Article: 8.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/24/2015] [Accepted: 08/20/2015] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Selection bias occurs when recruiters selectively enrol patients into the trial based on what the next treatment allocation is likely to be. This can occur even if appropriate allocation concealment is used if recruiters can guess the next treatment assignment with some degree of accuracy. This typically occurs in unblinded trials when restricted randomisation is implemented to force the number of patients in each arm or within each centre to be the same. Several methods to reduce the risk of selection bias have been suggested; however, it is unclear how often these techniques are used in practice. METHODS We performed a review of published trials which were not blinded to assess whether they utilised methods for reducing the risk of selection bias. We assessed the following techniques: (a) blinding of recruiters; (b) use of simple randomisation; (c) avoidance of stratification by site when restricted randomisation is used; (d) avoidance of permuted blocks if stratification by site is used; and (e) incorporation of prognostic covariates into the randomisation procedure when restricted randomisation is used. We included parallel group, individually randomised phase III trials published in four general medical journals (BMJ, Journal of the American Medical Association, The Lancet, and New England Journal of Medicine) in 2010. RESULTS We identified 152 eligible trials. Most trials (98%) provided no information on whether recruiters were blind to previous treatment allocations. Only 3% of trials used simple randomisation; 63% used some form of restricted randomisation, and 35% did not state the method of randomisation. Overall, 44% of trials were stratified by site of recruitment; 27% were not, and 29% did not report this information. Most trials that did stratify by site of recruitment used permuted blocks (58%), and only 15% reported using random block sizes. Many trials that used restricted randomisation also included prognostic covariates in the randomisation procedure (56%). CONCLUSIONS The risk of selection bias could not be ascertained for most trials due to poor reporting. Many trials which did provide details on the randomisation procedure were at risk of selection bias due to a poorly chosen randomisation methods. Techniques to reduce the risk of selection bias should be more widely implemented.
Collapse
Affiliation(s)
- Brennan C Kahan
- Pragmatic Clinical Trials Unit, Queen Mary University of London, E1 2AB, London, UK.
| | - Sunita Rehal
- MRC Clinical Trials Unit at UCL, WC2B 6NH, London, UK.
| | - Suzie Cro
- MRC Clinical Trials Unit at UCL, WC2B 6NH, London, UK.
| |
Collapse
|
20
|
Witkiewitz K, Finney JW, Harris AH, Kivlahan DR, Kranzler HR. Recommendations for the Design and Analysis of Treatment Trials for Alcohol Use Disorders. Alcohol Clin Exp Res 2015; 39:1557-70. [PMID: 26250333 PMCID: PMC4558228 DOI: 10.1111/acer.12800] [Citation(s) in RCA: 55] [Impact Index Per Article: 6.1] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2015] [Accepted: 05/30/2015] [Indexed: 12/14/2022]
Abstract
BACKGROUND Over the past 60 years, the view that "alcoholism" is a disease for which the only acceptable goal of treatment is abstinence has given way to the recognition that alcohol use disorders (AUDs) occur on a continuum of severity, for which a variety of treatment options are appropriate. However, because the available treatments for AUDs are not effective for everyone, more research is needed to develop novel and more efficacious treatments to address the range of AUD severity in diverse populations. Here we offer recommendations for the design and analysis of alcohol treatment trials, with a specific focus on the careful conduct of randomized clinical trials of medications and nonpharmacological interventions for AUDs. METHODS This paper provides a narrative review of the quality of published clinical trials and recommendations for the optimal design and analysis of treatment trials for AUDs. RESULTS Despite considerable improvements in the design of alcohol clinical trials over the past 2 decades, many studies of AUD treatments have used faulty design features and statistical methods that are known to produce biased estimates of treatment efficacy. CONCLUSIONS The published statistical and methodological literatures provide clear guidance on methods to improve clinical trial design and analysis. Consistent use of state-of-the-art design features and analytic approaches will enhance the internal and external validity of treatment trials for AUDs across the spectrum of severity. The ultimate result of this attention to methodological rigor is that better treatment options will be identified for patients with an AUD.
Collapse
Affiliation(s)
- Katie Witkiewitz
- Department of Psychology and Center on Alcoholism, Substance Abuse, and Addictions, University of New Mexico
| | - John W. Finney
- Center for Innovation to Implementation, VA Palo Alto Health Care System, Menlo Park, CA
| | - Alex H.S Harris
- VA Substance Use Disorder Quality Enhancement Research Initiative, VA Palo Alto Health Care System, Menlo Park, CA
| | - Daniel R. Kivlahan
- Veterans Health Administration, Washington, DC and Department of Psychiatry and Behavioral Sciences, University of Washington, Seattle
| | - Henry R. Kranzler
- Center for Studies of Addiction, Department of Psychiatry, University of Pennsylvania Perelman School of Medicine and VISN4 MIRECC, Philadelphia VAMC, Philadelphia, PA
| |
Collapse
|
21
|
Kennes LN, Rosenberger WF, Hilgers RD. Inference for blocked randomization under a selection bias model. Biometrics 2015; 71:979-84. [PMID: 26099068 DOI: 10.1111/biom.12334] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/01/2014] [Revised: 03/01/2015] [Accepted: 04/01/2015] [Indexed: 11/26/2022]
Abstract
We provide an asymptotic test to analyze randomized clinical trials that may be subject to selection bias. For normally distributed responses, and under permuted block randomization, we derive a likelihood ratio test of the treatment effect under a selection bias model. A likelihood ratio test of the presence of selection bias arises from the same formulation. We prove that the test is asymptotically chi-square on one degree of freedom. These results correlate well with the likelihood ratio test of Ivanova et al. (2005, Statistics in Medicine 24, 1537-1546) for binary responses, for which they established by simulation that the asymptotic distribution is chi-square. Simulations also show that the test is robust to departures from normality and under another randomization procedure. We illustrate the test by reanalyzing a clinical trial on retinal detachment.
Collapse
Affiliation(s)
- Lieven N Kennes
- Department of Medical Statistics, RWTH Aachen University, Pauwelsstrasse 30, 52074 Aachen, Germany
| | - William F Rosenberger
- Department of Medical Statistics, RWTH Aachen University, Pauwelsstrasse 30, 52074 Aachen, Germany.,Department of Statistics, George Mason University, 4400 University Drive, MS 4A7, Fairfax, Virginia 22030, U.S.A
| | - Ralf-Dieter Hilgers
- Department of Medical Statistics, RWTH Aachen University, Pauwelsstrasse 30, 52074 Aachen, Germany
| |
Collapse
|
22
|
Comparison of statistical and operational properties of subject randomization procedures for large multicenter clinical trial treating medical emergencies. Contemp Clin Trials 2015; 41:211-8. [PMID: 25638754 DOI: 10.1016/j.cct.2015.01.013] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/25/2014] [Revised: 01/20/2015] [Accepted: 01/21/2015] [Indexed: 01/28/2023]
Abstract
Large multicenter acute stroke trials demand a randomization procedure with a high level of treatment allocation randomness, an effective control on overall and within-site imbalances, and a minimized time delay of study treatment caused by the randomization procedure. Driven by the randomization algorithm design of A Study of the Efficacy and Safety of Activase (Alteplase) in Patients with Mild Stroke (PRISMS) (NCT02072226), this paper compares operational and statistical properties of different randomization algorithms in local, central, and step-forward randomization settings. Results show that the step-forward randomization with block urn design provides better performances over others. If the concern on the potential time delay is not serious and a central randomization system is available, the minimization method with an imbalance control threshold and a biased coin probability could be a better choice.
Collapse
|
23
|
Roberts JN, Karvonen C, Graham K, Weinfeld M, Joy AA, Koebel M, Morris D, Robson PJ, Johnston RN, Brockton NT. Biobanking in the Twenty-First Century: Driving Population Metrics into Biobanking Quality. ADVANCES IN EXPERIMENTAL MEDICINE AND BIOLOGY 2015; 864:95-114. [DOI: 10.1007/978-3-319-20579-3_8] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/07/2023]
|
24
|
Menadue C, Piper AJ, van 't Hul AJ, Wong KK. Non-invasive ventilation during exercise training for people with chronic obstructive pulmonary disease. Cochrane Database Syst Rev 2014; 2014:CD007714. [PMID: 24823712 PMCID: PMC10984247 DOI: 10.1002/14651858.cd007714.pub2] [Citation(s) in RCA: 27] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
BACKGROUND Exercise training as a component of pulmonary rehabilitation improves health-related quality of life (HRQL) and exercise capacity in people with chronic obstructive pulmonary disease (COPD). However, some individuals may have difficulty performing exercise at an adequate intensity. Non-invasive ventilation (NIV) during exercise improves exercise capacity and dyspnoea during a single exercise session. Consequently, NIV during exercise training may allow individuals to exercise at a higher intensity, which could lead to greater improvement in exercise capacity, HRQL and physical activity. OBJECTIVES To determine whether NIV during exercise training (as part of pulmonary rehabilitation) affects exercise capacity, HRQL and physical activity in people with COPD compared with exercise training alone or exercise training with sham NIV. SEARCH METHODS We searched the following databases between January 1987 and November 2013 inclusive: The Cochrane Airways Group specialised register of trials, AMED, CENTRAL, CINAHL, EMBASE, LILACS, MEDLINE, PEDro, PsycINFO and PubMed. SELECTION CRITERIA Randomised controlled trials that compared NIV during exercise training versus exercise training alone or exercise training with sham NIV in people with COPD were considered for inclusion in this review. DATA COLLECTION AND ANALYSIS Two review authors independently selected trials for inclusion in the review, extracted data and assessed risk of bias. Primary outcomes were exercise capacity, HRQL and physical activity; secondary outcomes were training intensity, physiological changes related to exercise training, dyspnoea, dropouts, adverse events and cost. MAIN RESULTS Six studies involving 126 participants who completed the study protocols were included. Most studies recruited participants with severe to very severe COPD (mean forced expiratory volume in one second (FEV1) ranged from 26% to 48% predicted). There was an increase in percentage change peak and endurance exercise capacity with NIV during training (mean difference in peak exercise capacity 17%, 95% confidence interval (CI) 7% to 27%, 60 participants, low-quality evidence; mean difference in endurance exercise capacity 59%, 95% CI 4% to 114%, 48 participants, low-quality evidence). However, there was no clear evidence of a difference between interventions for all other measures of exercise capacity. The results for HRQL assessed using the St George's Respiratory Questionnaire do not rule out an effect of NIV (total score mean 2.5 points, 95% CI -2.3 to 7.2, 48 participants, moderate-quality evidence). Physical activity was not assessed in any study. There was an increase in training intensity with NIV during training of 13% (95% CI 1% to 27%, 67 participants, moderate-quality evidence), and isoload lactate was lower with NIV (mean difference -0.97 mmol/L, 95% CI -1.58mmol/L to -0.36 mmol/L, 37 participants, moderate-quality evidence). The effect of NIV on dyspnoea or the number of dropouts between interventions was uncertain, although again results were imprecise. No adverse events and no information regarding cost were reported. Only one study blinded participants, whereas three studies used blinded assessors. Adequate allocation concealment was reported in four studies. AUTHORS' CONCLUSIONS The small number of included studies with small numbers of participants, as well as the high risk of bias within some of the included studies, limited our ability to draw strong evidence-based conclusions. Although NIV during lower limb exercise training may allow people with COPD to exercise at a higher training intensity and to achieve a greater physiological training effect compared with exercise training alone or exercise training with sham NIV, the effect on exercise capacity is unclear. Some evidence suggests that NIV during exercise training improves the percentage change in peak and endurance exercise capacity; however, these findings are not consistent across other measures of exercise capacity. There is no clear evidence that HRQL is better or worse with NIV during training. It is currently unknown whether the demonstrated benefits of NIV during exercise training are clinically worthwhile or cost-effective.
Collapse
Affiliation(s)
- Collette Menadue
- Royal Prince Alfred HospitalDepartment of Respiratory and Sleep MedicineMissenden RoadCamperdownNSWAustralia2050
| | - Amanda J Piper
- Royal Prince Alfred HospitalDepartment of Respiratory and Sleep MedicineMissenden RoadCamperdownNSWAustralia2050
- Woolcock Institute of Medical Research431 Glebe Point RoadGlebeNSWAustralia2037
| | | | - Keith K Wong
- Royal Prince Alfred HospitalDepartment of Respiratory and Sleep MedicineMissenden RoadCamperdownNSWAustralia2050
- Woolcock Institute of Medical Research431 Glebe Point RoadGlebeNSWAustralia2037
| | | |
Collapse
|
25
|
Tamm M, Cramer E, Kennes LN, Heussen N. Influence of selection bias on the test decision. A simulation study. Methods Inf Med 2011; 51:138-43. [PMID: 22101391 DOI: 10.3414/me11-01-0043] [Citation(s) in RCA: 16] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/18/2011] [Accepted: 09/09/2011] [Indexed: 11/09/2022]
Abstract
BACKGROUND Selection bias arises in clinical trials by reason of selective assignment of patients to treatment groups. Even in randomized clinical trials with allocation concealment this phenomenon can occur if future assignments can be predicted due to knowledge of former allocations. OBJECTIVES Considering unmasked randomized clinical trials with allocation concealment the impact of selection bias on type I error rate under permuted block randomization is investigated. We aimed to extend the existing research into this topic by including practical assumptions concerning misclassification of patient characteristics to get an estimate of type I error close to clinical routine. To establish an upper bound for the type I error rate different biasing strategies of the investigator are compared first. In addition, the aspect of patient availability is considered. METHODS To evaluate the influence of selection bias on type I error rate under several practical situations, different block sizes, selection effects, biasing strategies and success rates of patient classification were simulated using SAS. RESULTS Type I error rate exceeds 5 percent significance level; it reaches values up to 21 percent. More cautious biasing strategies and misclassification of patient characteristics may diminish but cannot eliminate selection bias. The number of screened patients is about three times larger than the needed number for the trial. CONCLUSIONS Even in unmasked randomized clinical trials using permuted block randomization with allocation concealment the influence of selection bias must not be disregarded evaluating the test decision. It should be incorporated when designing and reporting a clinical trial.
Collapse
Affiliation(s)
- M Tamm
- RWTH Aachen University, Department of Medical Statistics, Pauwelsstraße 30, 52074 Aachen, Germany.
| | | | | | | |
Collapse
|
26
|
Different methods of allocation to groups in randomized trials are associated with different levels of bias. A meta-epidemiological study. J Clin Epidemiol 2011; 64:1070-5. [DOI: 10.1016/j.jclinepi.2010.12.018] [Citation(s) in RCA: 32] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/27/2010] [Revised: 12/02/2010] [Accepted: 12/10/2010] [Indexed: 11/19/2022]
|
27
|
Kennes LN, Cramer E, Hilgers RD, Heussen N. The impact of selection bias on test decisions in randomized clinical trials. Stat Med 2011; 30:2573-81. [PMID: 21717489 DOI: 10.1002/sim.4279] [Citation(s) in RCA: 15] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2010] [Accepted: 04/01/2011] [Indexed: 11/06/2022]
Abstract
Selection bias affects the evaluation of clinical trials, for example, by elevating type I error rate. We investigated the effect of selection bias on type I error rate considering permuted block randomization. We also considered stratified randomization in general and for the special case of multicenter clinical trials, where we incorporated preferences of the investigator in our approach. Finally, the effect of underrunning is modeled, that is, where the randomization list exceeds the actual number of patients taking part in the trial. For all situations, we illustrate and discuss the impact of selection bias on type I error rate.
Collapse
Affiliation(s)
- Lieven Nils Kennes
- RWTH Aachen University, Department of Medical Statistics, Pauwelsstrasse 30, Aachen 52074, Germany.
| | | | | | | |
Collapse
|
28
|
Reporting of treatment effects from randomized trials: A plea for multivariable risk ratios. Contemp Clin Trials 2011; 32:399-402. [DOI: 10.1016/j.cct.2010.12.011] [Citation(s) in RCA: 20] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/27/2010] [Accepted: 12/21/2010] [Indexed: 11/22/2022]
|
29
|
Zhao W, Weng Y. A simplified formula for quantification of the probability of deterministic assignments in permuted block randomization. J Stat Plan Inference 2011; 141:474-478. [PMID: 21057655 DOI: 10.1016/j.jspi.2010.06.023] [Citation(s) in RCA: 10] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
Abstract
Open label and single blinded randomized controlled clinical trials are vulnerable to selection bias when the next treatment assignment is predictable based on the randomization algorithm and the preceding assignment history. While treatment predictability is an issue for all constrained randomization algorithms, deterministic assignments are unique to permuted block randomization. Deterministic assignments may lead to treatment predictability with certainty and selection bias, which could inflate the type I error and hurts the validity of trial results. It is important to accurately evaluate the probability of deterministic assignments in permuted block randomization, so proper protection measures can be implemented. For trials with number of treatment arms T = 2 and a balance block size B = 2m, Matts and Lachin indicated that the probability of deterministic assignment is 1m+1. For more general situations, with T ≥ 2 and a block size B=∑j=1Tmj, Dupin-Spriet provided a formula, which can be written as 1B∑j=1T∑i=1mj∏k=1imj-k+1B-k+1. This formula involves extensive calculation in evaluation. In this paper, we simplified this formula to 1B∑j=1TmjB-mj+1 for general scenarios and 1B-m+1 for trials with a balanced allocation. Through mathematical induction we show the equivalence of the formulas. While the new formula is numerically equivalent to Dupin-Spriet's formula, the simple format not only is easier for evaluation, but also is clearer in describing the impact of parameters T and m(i) on the probability of deterministic assignments.
Collapse
Affiliation(s)
- Wenle Zhao
- Department of Medicine, Medical University of South Carolina, Charleston, SC, USA
| | | |
Collapse
|
30
|
Berger VW, Knoll D. Levels of evidence: further insight on Gugiu and Gugiu. Eval Health Prof 2010; 34:124-6; discussion 127-30. [PMID: 21177639 DOI: 10.1177/0163278710391466] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/10/2023]
Affiliation(s)
- Vance W Berger
- Biometry Research Group, National Cancer Institute, Bethesda, MD 20892, USA.
| | | |
Collapse
|
31
|
Berger VW. Randomization, permuted blocks, masking, allocation concealment, and selection bias in the Tobacco Quit Line Study. Contemp Clin Trials 2010; 31:201. [PMID: 20219696 DOI: 10.1016/j.cct.2010.02.004] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/19/2010] [Accepted: 02/23/2010] [Indexed: 01/10/2023]
|
32
|
Balance algorithm for cluster randomized trials. BMC Med Res Methodol 2008; 8:65. [PMID: 18844993 PMCID: PMC2588445 DOI: 10.1186/1471-2288-8-65] [Citation(s) in RCA: 81] [Impact Index Per Article: 5.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2008] [Accepted: 10/09/2008] [Indexed: 11/25/2022] Open
Abstract
Background Within cluster randomized trials no algorithms exist to generate a full enumeration of a block randomization, balancing for covariates across treatment arms. Furthermore, often for practical reasons multiple blocks are required to fully randomize a study, which may not have been well balanced within blocks. Results We present a convenient and easy to use randomization tool to undertake allocation concealed block randomization. Our algorithm highlights allocations that minimize imbalance between treatment groups across multiple baseline covariates. We demonstrate the algorithm using a cluster randomized trial in primary care (the PRE-EMPT Study) and show that the software incorporates a trade off between independent random allocations that were likely to be imbalanced, and predictable deterministic approaches that would minimise imbalance. We extend the methodology of single block randomization to allocate to multiple blocks conditioning on previous allocations. Conclusion The algorithm is included as Additional file 1 and we advocate its use for robust randomization within cluster randomized trials.
Collapse
|
33
|
Austin PC. A critical appraisal of propensity-score matching in the medical literature between 1996 and 2003. Stat Med 2008; 27:2037-49. [PMID: 18038446 DOI: 10.1002/sim.3150] [Citation(s) in RCA: 837] [Impact Index Per Article: 52.3] [Reference Citation Analysis] [Abstract] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/16/2022]
Abstract
Propensity-score methods are increasingly being used to reduce the impact of treatment-selection bias in the estimation of treatment effects using observational data. Commonly used propensity-score methods include covariate adjustment using the propensity score, stratification on the propensity score, and propensity-score matching. Empirical and theoretical research has demonstrated that matching on the propensity score eliminates a greater proportion of baseline differences between treated and untreated subjects than does stratification on the propensity score. However, the analysis of propensity-score-matched samples requires statistical methods appropriate for matched-pairs data. We critically evaluated 47 articles that were published between 1996 and 2003 in the medical literature and that employed propensity-score matching. We found that only two of the articles reported the balance of baseline characteristics between treated and untreated subjects in the matched sample and used correct statistical methods to assess the degree of imbalance. Thirteen (28 per cent) of the articles explicitly used statistical methods appropriate for the analysis of matched data when estimating the treatment effect and its statistical significance. Common errors included using the log-rank test to compare Kaplan-Meier survival curves in the matched sample, using Cox regression, logistic regression, chi-squared tests, t-tests, and Wilcoxon rank sum tests in the matched sample, thereby failing to account for the matched nature of the data. We provide guidelines for the analysis and reporting of studies that employ propensity-score matching.
Collapse
Affiliation(s)
- Peter C Austin
- Institute for Clinical Evaluative Sciences, Toronto, Ont., Canada.
| |
Collapse
|
34
|
Hansen BB, Bowers J. Covariate Balance in Simple, Stratified and Clustered Comparative Studies. Stat Sci 2008. [DOI: 10.1214/08-sts254] [Citation(s) in RCA: 274] [Impact Index Per Article: 17.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
|
35
|
Friede T. A Review of: “Selection Bias and Covariate Imbalances in Randomized Clinical Trials, by V. W. Berger”. J Biopharm Stat 2007. [DOI: 10.1080/10543400600795053] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
Affiliation(s)
- Tim Friede
- a Biostatistics and Statistical Reporting Novartis Pharma AG CH-4002, Basel , Switzerland
| |
Collapse
|
36
|
Abstract
We consider the problem of testing for independence against the consistent superiority of one treatment over another when the response variable is binary and is compared across two treatments in each of several strata. Specifically, we consider the randomized clinical trial setting. A number of issues arise in this context. First, should tables be combined if there are small or zero margins? Second, should one assume a common odds ratio across strata? Third, if the odds ratios differ across strata, then how does the standard test (based on a common odds ratio) perform? Fourth, are there other analyzes that are more appropriate for handling a situation in which the odds ratios may differ across strata? In addressing these issues we find that the frequently used Cochran-Mantel-Haenszel test may have a poor power profile, despite being optimal when the odds ratios are common. We develop novel tests that are analogous to the Smirnov, modified Smirnov, convex hull, and adaptive tests that have been proposed for ordered categorical data.
Collapse
Affiliation(s)
- Vance W Berger
- University of Maryland Baltimore County and National Cancer Institute, Executive Plaza North, Suite 3131, Bethesda, MD 20892-7354, USA
| | | | | |
Collapse
|
37
|
|
38
|
Berger VW. Allocation concealment and blinding: when ignorance is bliss. Med J Aust 2005; 183:165; author reply 166. [PMID: 16053426 DOI: 10.5694/j.1326-5377.2005.tb06974.x] [Citation(s) in RCA: 21] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2005] [Accepted: 05/05/2005] [Indexed: 11/17/2022]
|
39
|
Altman DG. Comment. Biom J 2005. [DOI: 10.1002/bimj.200510108] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
|
40
|
|
41
|
|