Quantifying the Magnitude of Baseline Covariate Imbalances Resulting from Selection Bias in Randomized Clinical Trials

A deprescribing programme aimed to optimise blood glucose-lowering medication in older people with type 2 diabetes mellitus, the OMED2-study: the study protocol for a randomised controlled trial

BACKGROUND

Older patients with type 2 diabetes mellitus (T2D) have an increased risk of hypoglycaemic episodes when using sulphonylureas or insulin. In the Netherlands, guidelines exist for reducing glucose-lowering medication in older patients. However, evidence is lacking that a medication reduction in older patients can be safely pursued. Here, we will examine if promoting the deprescribing of insulin/sulphonylureas with a deprescribing programme (DPP) in general practice affects T2D-complications in older overtreated patients.

METHODS

We will perform a 1:1 cluster randomised controlled trial in 86 general practices in the Netherlands. The DPP will consist of education sessions with general practitioners and practice nurses about reducing glucose-lowering medication in older patients (≥ 70 years). Topics of the sessions include the necessity of deprescribing, tools to initiate deprescribing and strategies to discuss deprescribing with patients (shared decision making). The DPP further includes a support programme with practice visits. The study will employ a selection tool to identify possibly overtreated older patients from the electronic medical records of the general practitioner. Eligibility for enrolment in the study will be based on HbA1c targets indicated by the Dutch guidelines, which depend on age, diabetes duration, presence of frailty, and life expectancy. The control group will provide usual care. We aim to include 406 patients. The follow-up period will be 2 years. For the primary outcome, the effect of the DPP on T2D-complications will be assessed by counting the cumulative incidence of events related to under- and overtreatment in T2D as registered in the electronic medical records. We shall perform an intention-to-treat analysis and an analysis including only patients for whom deprescribing was initiated. The implementation of the DPP in general practice will be evaluated quantitatively and qualitatively using the Extended Normalisation Process Theory (ENPT) and the Reach, Efficacy - Adoption, Implementation and Maintenance (RE-AIM) model. Other secondary outcomes include quality of life, cognitive functioning, events related to overtreatment or undertreatment, biomarkers of health, amount of blood glucose-lowering medication prescriptions, and cost-effectiveness.

DISCUSSION

This study will provide insight into the safety and feasibility of a programme aimed at deprescribing sulphonylureas/insulin in older people with T2D who are treated in general practice.

TRIAL REGISTRATION

ISRCTN Registry, ISRCTN50008265 , registered 09 March, 2023.

Prevalence and implications of significance testing for baseline covariate imbalance in randomised cancer clinical trials: The Table 1 Fallacy

BACKGROUND

The 'Table 1 Fallacy' refers to the unsound use of significance testing for comparing the distributions of baseline variables between randomised groups to draw erroneous conclusions about balance or imbalance. We performed a cross-sectional study of the Table 1 Fallacy in phase III oncology trials.

METHODS

From ClinicalTrials.gov, 1877 randomised trials were screened. Multivariable logistic regressions evaluated predictors of the Table 1 Fallacy.

RESULTS

A total of 765 randomised controlled trials involving 553,405 patients were analysed. The Table 1 Fallacy was observed in 25% of trials (188 of 765), with 3% of comparisons deemed significant (59 of 2353), approximating the typical 5% type I error assertion probability. Application of trial-level multiplicity corrections reduced the rate of significant findings to 0.3% (six of 2345 tests). Factors associated with lower odds of the Table 1 Fallacy included industry sponsorship (adjusted odds ratio [aOR] 0.29, 95% confidence interval [CI] 0.18-0.47; multiplicity-corrected P < 0.0001), larger trial size (≥795 versus <280 patients; aOR 0.32, 95% CI 0.19-0.53; multiplicity-corrected P = 0.0008), and publication in a European versus American journal (aOR 0.06, 95% CI 0.03-0.13; multiplicity-corrected P < 0.0001).

CONCLUSIONS

This study highlights the persistence of the Table 1 Fallacy in contemporary oncology randomised controlled trials, with one of every four trials testing for baseline differences after randomisation. Significance testing is a suboptimal method for identifying unsound randomisation procedures and may encourage misleading inferences. Journal-level enforcement is a possible strategy to help mitigate this fallacy.

A SuperLearner-enforced approach for the estimation of treatment effect in pediatric trials

Background

Randomized Clinical Trials (RCT) represent the gold standard among scientific evidence. RCTs are tailored to control selection bias and the confounding effect of baseline characteristics on the effect of treatment. However, trial conduction and enrolment procedures could be challenging, especially for rare diseases and paediatric research. In these research frameworks, the treatment effect estimation could be compromised. A potential countermeasure is to develop predictive models on the probability of the baseline disease based on previously collected observational data. Machine learning (ML) algorithms have recently become attractive in clinical research because of their flexibility and improved performance compared to standard statistical methods in developing predictive models.

Objective

This manuscript proposes an ML-enforced treatment effect estimation procedure based on an ensemble SuperLearner (SL) approach, trained on historical observational data, to control the confounding effect.

Methods

The REnal SCarring Urinary infEction trial served as a motivating example. Historical observational study data have been simulated through 10,000 Monte Carlo (MC) runs. Hypothetical RCTs have been also simulated, for each MC run, assuming different treatment effects of antibiotics combined with steroids. For each MC simulation, the SL tool has been applied to the simulated observational data. Furthermore, the average treatment effect (ATE), has been estimated on the trial data and adjusted for the SL predicted probability of renal scar.

Results

The simulation results revealed an increased power in ATE estimation for the SL-enforced estimation compared to the unadjusted estimates for all the algorithms composing the ensemble SL.

A review found small variable blocking schemes may not protect against selection bias in randomized controlled trials

OBJECTIVE

Blocking is associated with prediction of the allocation sequence and subversion. This paper explores if blocking strategies lead to an increase in baseline age heterogeneity (a marker for potential subversion) and, whether the use of blocking is changing over time.

STUDY DESIGN AND SETTINGS

The British Medical Journal, Journal of the American Medical Association, The Lancet and the New England Journal of Medicine were hand searched to identify open RCTs published in January between 2001 and 2020. To explore heterogeneity of baseline age meta-analyses were performed on trials implementing blocking, minimization, and simple randomization.

RESULTS

One hundred seventy-nine open RCTs were identified: nine (5.0%) undertook simple randomization, 104 (58.1%) blocking, 25 (13.9%) minimization, and one (0.6%) both. Baseline age heterogeneity of 24% (P= 0.02) was observed in all trials implementing blocking, 62% (P = 0.001) in trials implementing a fixed block of four, 40% (P = 0.07) implementing variable blocks including a 2 and 0% for both simple randomization and minimization. Small block sizes are implemented in modern trials.

CONCLUSION

Variable block sizes including two are associated with subversion and should not be implemented. If center only stratification is necessary, it should be used alongside larger blocking schemes. Authors should consider alternative methods to restrict randomization.

Randomization-based inference in the presence of selection bias

For the analysis of clinical trials, the study participants are usually assumed to be representative sample of a target population. This assumption is rarely fulfilled in clinical trials, and particularly not if the sample size is small. In addition, covariate imbalances may affect the trial. Randomization tests provide a nonparametric analysis method of the treatment effect that does not rely on population-based assumptions. We propose a nonparametric statistical model that yields a formal basis for randomization tests. We adapt the model for the presence of covariate imbalance in the form of selection bias and investigate the effects of bias on the rejection probability of the randomization test using Monte Carlo simulations. Finally, we show that ancillary statistics can be used to control for the influence of bias. We show that covariate imbalance leads to an inflation of the type I error probability. The proposed nonparametric model allows for the use of ancillary statistics that yield an unbiased adjusted randomization test.

The Use of (Network) Meta-Analysis in Clinical Oncology

Meta-analysis is important in oncological research to provide a more reliable answer to a clinical research question that was assessed in multiple studies but with inconsistent results. Pair-wise meta-analysis can be applied when comparing two treatments at once, whereas it is possible to compare multiple treatments at once with network meta-analysis (NMA). After careful systematic review of the literature and quality assessment of the identified studies, there are several assumptions in the use of meta-analysis. First, the added value of meta-analysis should be evaluated by examining the comparability of study populations. Second, the appropriate comparator in meta-analysis should be chosen according to the types of comparisons made in individual studies: (1) Experimental and comparator arms are different treatments (A vs. B); (2) Substitution of a conventional treatment by an experimental treatment (A+B vs. A+C); or (3) Addition of an experimental treatment (A+B vs. B). Ideally there is one common comparator treatment, but when there are multiple common comparators, the most efficacious comparator is preferable. Third, treatments can only be adequately pooled in meta-analysis or merged into one treatment node in NMA when considering likewise mechanism of action and similar setting in which treatment is indicated. Fourth, for both pair-wise meta-analysis and NMA, adequate assessment of heterogeneity should be performed and sub-analysis and sensitivity analysis can be applied to objectify a possible confounding factor. Network inconsistency, as statistical manifestation of violating the transitivity assumption, can best be evaluated by node-split modeling. NMA has advantages over pair-wise meta-analysis, such as clarification of inconsistent outcomes from multiple studies including multiple common comparators and indirect effect calculation of missing direct comparisons between important treatments. Also, NMA can provide increased statistical power and cross-validation of the observed treatment effect of weak connections with reasonable network connectivity and sufficient sample-sizes. However, inappropriate use of NMA can cause misleading results, and may emerge when there is low network connectivity, and therefore low statistical power. Furthermore, indirect evidence is still observational and should be interpreted with caution. NMA should therefore preferably be conducted and interpreted by both expert clinicians in the field and an experienced statistician. Finally, the use of meta-analysis can be extended to other areas, for example the identification of prognostic and predictive factors. Also, the integration of evidence from both meta-analysis and expert opinion can improve the construction of prognostic models in real-world databases.

Randomization in survival studies: An evaluation method that takes into account selection and chronological bias

The random allocation of patients to treatments is a crucial step in the design and conduct of a randomized controlled trial. For this purpose, a variety of randomization procedures is available. In the case of imperfect blinding, the extent to which a randomization procedure forces balanced group sizes throughout the allocation process affects the predictability of allocations. As a result, some randomization procedures perform superior with respect to selection bias, whereas others are less susceptible to chronological bias. The choice of a suitable randomization procedure therefore depends on the expected risk for selection and chronological bias within the particular study in question. To enable a sound comparison of different randomization procedures, we introduce a model for the combined effect of selection and chronological bias in randomized studies with a survival outcome. We present an evaluation method to quantify the influence of bias on the test decision of the log-rank test in a randomized parallel group trial with a survival outcome. The effect of selection and chronological bias and the dependence on the study setting are illustrated in a sensitivity analysis. We conclude with a case study to showcase the application of our model for comparing different randomization procedures in consideration of the expected type I error probability.

Influence of blinding on treatment effect size estimate in randomized controlled trials of oral health interventions

Background

Recent methodologic evidence suggests that lack of blinding in randomized trials can result in under- or overestimation of the treatment effect size. The objective of this study is to quantify the extent of bias associated with blinding in randomized controlled trials of oral health interventions.

Methods

We selected all oral health meta-analyses that included a minimum of five randomized controlled trials. We extracted data, in duplicate, related to nine blinding-related criteria, namely: patient blinding, assessor blinding, care-provider blinding, investigator blinding, statistician blinding, blinding of both patients and assessors, study described as “double blind”, blinding of patients, assessors, and care providers concurrently, and the appropriateness of blinding. We quantified the impact of bias associated with blinding on the magnitude of effect size using a two-level meta-meta-analytic approach with a random effects model to allow for intra- and inter-meta-analysis heterogeneity.

Results

We identified 540 randomized controlled trials, included in 64 meta-analyses, analyzing data from 137,957 patients. We identified significantly larger treatment effect size estimates in trials that had inadequate patient blinding (difference in treatment effect size = 0.12; 95% CI: 0.00 to 0.23), lack of blinding of both patients and assessors (difference = 0.19; 95% CI: 0.06 to 0.32), and lack of blinding of patients, assessors, and care-providers concurrently (difference = 0.14; 95% CI: 0.03 to 0.25). In contrast, assessor blinding (difference = 0.06; 95% CI: -0.06 to 0.18), caregiver blinding (difference = 0.02; 95% CI: -0.04 to 0.09), principal-investigator blinding (difference = − 0.02; 95% CI: -0.10 to 0.06), describing a trial as “double-blind” (difference = 0.09; 95% CI: -0.05 to 0.22), and lack of an appropriate method of blinding (difference = 0.06; 95% CI: -0.06 to 0.18) were not associated with over- or underestimated treatment effect size.

Conclusions

We found significant differences in treatment effect size estimates between oral health trials based on lack of patient and assessor blinding. Treatment effect size estimates were 0.19 and 0.14 larger in trials with lack of blinding of both patients and assessors and blinding of patients, assessors, and care-providers concurrently. No significant differences were identified in other blinding criteria. Investigators of oral health systematic reviews should perform sensitivity analyses based on the adequacy of blinding in included trials.

Electronic supplementary material

The online version of this article (10.1186/s12874-018-0491-0) contains supplementary material, which is available to authorized users.

The impact of selection bias in randomized multi-arm parallel group clinical trials

The impact of selection bias on the results of clinical trials has been analyzed extensively for trials of two treatments, yet its impact in multi-arm trials is still unknown. In this paper, we investigate selection bias in multi-arm trials by its impact on the type I error probability. We propose two models for selection bias, so-called biasing policies, that both extend the classic guessing strategy by Blackwell and Hodges. We derive the distribution of the F-test statistic under the misspecified outcome model and provide a formula for the type I error probability under selection bias. We apply the presented approach to quantify the influence of selection bias in multi-arm trials with increasing number of treatment groups using a permuted block design for different assumptions and different biasing strategies. Our results confirm previous findings that smaller block sizes lead to a higher proportion of sequences with inflated type I error probability. Astonishingly, our results also show that the proportion of sequences with inflated type I error probability remains constant when the number of treatment groups is increased. Realizing that the impact of selection bias cannot be completely eliminated, we propose a bias adjusted statistical model and show that the power of the statistical test is only slightly deflated for larger block sizes.

The effectiveness of manual therapy in treating cervicogenic dizziness: a systematic review

[Purpose] This review provides an evaluation of the evidence for the effectiveness of using manual therapy to treat cervicogenic dizziness. [Subjects and Methods] The literature was systematically searched on the May 2, 2016 using the following online databases: Medline, EMBASE, CINAHL and PEDro. This review included randomised controlled trials and compared the efficacy of manual therapy for the treatment of cervicogenic dizziness, compared to other types of intervention. This study measured changes based on dizziness intensity and frequency. [Results] The primary search found 30 articles, but only four articles met the inclusion criteria. Assessment of methodological quality was performed by two researchers using the PEDro scale. The level of evidence was determined using a recognised grading scale. Three out of the four articles were deemed to have high methodological quality, while the fourth was rated as moderate quality. The attributed level of evidence was moderate (level 2). [Conclusion] Manual therapy is potentially effective for managing cervicogenic dizziness. However, due to the heterogeneity of the results and techniques and the low number of studies, further research is recommended to provide conclusive evidence.

Cognitive behavioural therapy and pain coping skills training for osteoarthritis knee pain management: a systematic review

[Purpose] To investigate the effectiveness of cognitive-behavioural therapy (CBT) and pain coping skills training (PCST) on pain level in adults with osteoarthritis of the knee (KOA) in comparison with usual care. [Subjects and Methods] Five databases were systematically searched for relevant randomised controlled trials (RCTs) according to the selected eligibility criteria (inception to June 7, 2016). PEDro scale was used to assess the validity of included studies. [Results] Four studies met the inclusion criteria and all studies had high methodological quality. The total number of participants was 665 across the four included studies. All participants had been diagnosed with KOA (clinical and radiographic evaluation). The majority of the population were female (71.13%). The interventions utilised were: CBT, CBT for insomnia (CBT-I), PCST, and PCST combined with behavioral weight management (BWM) in one study and with exercise in another study. The outcome measure for pain in KOA was the WOMAC scale. Overall, three studies have reported clinical improvement in KOA pain perception after 12-month follow-up time point. [Conclusion] This review has recognised the need for future studies that have rigorous methodological quality, and investigate the effect of mutual CBT and PCST protocols on KOA pain and pain-related functional and psychological abilities.

Methodological Aspects in Studies Based on Clinical Routine Data

Randomized controlled clinical trials are regarded as the gold standard for comparing different clinical interventions, but generally their conduct is operationally cumbersome, time-consuming, and expensive. Studies and investigations based on clinical routine data on the contrary utilize existing data acquired under real-life conditions and are increasingly popular among practitioners. In this paper, methodological aspects of studies based on clinical routine data are discussed. Important limitations and considerations as well as unique strengths of these types of studies are indicated and exemplarily demonstrated in a recent real-case study based on clinical routine data. In addition two simulation studies reveal the impact of bias in studies based on clinical routine data on the type I error rate and false decision rate in favor of the inferior intervention. It is concluded that correctly analyzing clinical routine data yields a valuable addition to clinical research; however, as a result of a lack of statistical foundation, internal validity, and comparability, generalizing results and inferring properties derived from clinical routine data to all patients of interest has to be considered with extreme caution.

FUNDING

Grünenthal GmbH.

Assessing the impact of selection bias on test decisions in trials with a time-to-event outcome

If past treatment assignments are unmasked, selection bias may arise even in randomized controlled trials. The impact of such bias can be measured by considering the type I error probability. In case of a normally distributed outcome, there already exists a model accounting for selection bias that permits calculating the corresponding type I error probabilities. To model selection bias for trials with a time‐to‐event outcome, we introduce a new biasing policy for exponentially distributed data. Using this biasing policy, we derive an exact formula to compute type I error probabilities whenever an F‐test is performed and no observations are censored. Two exemplary settings, with and without random censoring, are considered in order to illustrate how our results can be applied to compare distinct randomization procedures with respect to their performance in the presence of selection bias. © 2017 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.

Mechanisms and direction of allocation bias in randomised clinical trials

BACKGROUND

Selective allocation of patients into the compared groups of a randomised trial may cause allocation bias, but the mechanisms behind the bias and its directionality are incompletely understood. We therefore analysed the mechanisms and directionality of allocation bias in randomised clinical trials.

METHODS

Two systematic reviews and a theoretical analysis. We conducted one systematic review of empirical studies of motives/methods for deciphering patient allocation sequences; and another review of methods publications commenting on allocation bias. We theoretically analysed the mechanisms of allocation bias and hypothesised which main factors predicts its direction.

RESULTS

Three empirical studies addressed motives/methods for deciphering allocation sequences. Main motives included ensuring best care for patients and ensuring best outcome for the trial. Main methods included various manipulations with randomisation envelopes. Out of 57 methods publications 11 (19 %) mentioned explicitly that allocation bias can go in either direction. We hypothesised that the direction of allocation bias is mainly decided by the interaction between the patient allocators' motives and treatment preference.

CONCLUSION

Inadequate allocation concealment may exaggerate treatment effects in some trials while underestimate effects in others. Our hypothesis provides a theoretical overview of the main factors responsible for the direction of allocation bias.

Compression Stockings for Preventing the Postthrombotic Syndrome in Patients with Deep Vein Thrombosis

OBJECTIVE

We conducted a systematic review and meta-analysis to address benefits and harms of using elastic compression stockings after lower-extremity deep vein thrombosis.

METHODS

We searched 7 electronic databases through January 15, 2015, including randomized controlled trials (RCTs)/quasi-randomized trials reporting on elastic compression stocking efficacy on postthrombotic syndrome incidence, recurrent venous thromboembolism, mortality, and acute pain after deep vein thrombosis. Two reviewers independently screened records, extracted data, assessed risk of bias, and assessed confidence in effect estimates using Grading of Recommendations Assessment, Development, and Evaluation methodology. We applied random-effects meta-analysis models.

RESULTS

We included 5 RCTs (n = 1418) reporting on postthrombotic syndrome. The hazard ratio (HR) for postthrombotic syndrome with elastic compression stockings was 0.69 (95% confidence interval [CI], 0.47-1.02). We have very low confidence in this estimate due to heterogeneity and inclusion of unblinded studies at high risk of bias. Excluding high risk of bias studies, a single large RCT at low risk of bias provided moderate-quality evidence of no effect on postthrombotic syndrome (HR 1.00; 95% CI, 0.81-1.24). Moderate-quality evidence including all 5 studies suggests no effect of elastic compression stockings on recurrent venous thromboembolism (relative risk [RR] 0.88; 95% CI, 0.63-1.24) or mortality (RR 1.00; 95% CI, 0.73-1.37, 5 studies). Moderate-quality evidence from one large RCT does not suggest effect on acute pain after deep vein thrombosis.

CONCLUSIONS

The highest-quality evidence available suggests no effect of elastic compression stockings on postthrombotic syndrome or pain relief, from a single large RCT. However, results for preventing postthrombotic syndrome differ substantially across studies, and future guideline updates should reflect uncertainty about treatment effects. Elastic compression stockings are unlikely to prevent death or recurrent venous thromboembolism.

Risk of selection bias in randomised trials

BACKGROUND

Selection bias occurs when recruiters selectively enrol patients into the trial based on what the next treatment allocation is likely to be. This can occur even if appropriate allocation concealment is used if recruiters can guess the next treatment assignment with some degree of accuracy. This typically occurs in unblinded trials when restricted randomisation is implemented to force the number of patients in each arm or within each centre to be the same. Several methods to reduce the risk of selection bias have been suggested; however, it is unclear how often these techniques are used in practice.

METHODS

We performed a review of published trials which were not blinded to assess whether they utilised methods for reducing the risk of selection bias. We assessed the following techniques: (a) blinding of recruiters; (b) use of simple randomisation; (c) avoidance of stratification by site when restricted randomisation is used; (d) avoidance of permuted blocks if stratification by site is used; and (e) incorporation of prognostic covariates into the randomisation procedure when restricted randomisation is used. We included parallel group, individually randomised phase III trials published in four general medical journals (BMJ, Journal of the American Medical Association, The Lancet, and New England Journal of Medicine) in 2010.

RESULTS

We identified 152 eligible trials. Most trials (98%) provided no information on whether recruiters were blind to previous treatment allocations. Only 3% of trials used simple randomisation; 63% used some form of restricted randomisation, and 35% did not state the method of randomisation. Overall, 44% of trials were stratified by site of recruitment; 27% were not, and 29% did not report this information. Most trials that did stratify by site of recruitment used permuted blocks (58%), and only 15% reported using random block sizes. Many trials that used restricted randomisation also included prognostic covariates in the randomisation procedure (56%).

CONCLUSIONS

The risk of selection bias could not be ascertained for most trials due to poor reporting. Many trials which did provide details on the randomisation procedure were at risk of selection bias due to a poorly chosen randomisation methods. Techniques to reduce the risk of selection bias should be more widely implemented.

Recommendations for the Design and Analysis of Treatment Trials for Alcohol Use Disorders

BACKGROUND

Over the past 60 years, the view that "alcoholism" is a disease for which the only acceptable goal of treatment is abstinence has given way to the recognition that alcohol use disorders (AUDs) occur on a continuum of severity, for which a variety of treatment options are appropriate. However, because the available treatments for AUDs are not effective for everyone, more research is needed to develop novel and more efficacious treatments to address the range of AUD severity in diverse populations. Here we offer recommendations for the design and analysis of alcohol treatment trials, with a specific focus on the careful conduct of randomized clinical trials of medications and nonpharmacological interventions for AUDs.

METHODS

This paper provides a narrative review of the quality of published clinical trials and recommendations for the optimal design and analysis of treatment trials for AUDs.

RESULTS

Despite considerable improvements in the design of alcohol clinical trials over the past 2 decades, many studies of AUD treatments have used faulty design features and statistical methods that are known to produce biased estimates of treatment efficacy.

CONCLUSIONS

The published statistical and methodological literatures provide clear guidance on methods to improve clinical trial design and analysis. Consistent use of state-of-the-art design features and analytic approaches will enhance the internal and external validity of treatment trials for AUDs across the spectrum of severity. The ultimate result of this attention to methodological rigor is that better treatment options will be identified for patients with an AUD.

Inference for blocked randomization under a selection bias model

We provide an asymptotic test to analyze randomized clinical trials that may be subject to selection bias. For normally distributed responses, and under permuted block randomization, we derive a likelihood ratio test of the treatment effect under a selection bias model. A likelihood ratio test of the presence of selection bias arises from the same formulation. We prove that the test is asymptotically chi-square on one degree of freedom. These results correlate well with the likelihood ratio test of Ivanova et al. (2005, Statistics in Medicine 24, 1537-1546) for binary responses, for which they established by simulation that the asymptotic distribution is chi-square. Simulations also show that the test is robust to departures from normality and under another randomization procedure. We illustrate the test by reanalyzing a clinical trial on retinal detachment.

Comparison of statistical and operational properties of subject randomization procedures for large multicenter clinical trial treating medical emergencies

Large multicenter acute stroke trials demand a randomization procedure with a high level of treatment allocation randomness, an effective control on overall and within-site imbalances, and a minimized time delay of study treatment caused by the randomization procedure. Driven by the randomization algorithm design of A Study of the Efficacy and Safety of Activase (Alteplase) in Patients with Mild Stroke (PRISMS) (NCT02072226), this paper compares operational and statistical properties of different randomization algorithms in local, central, and step-forward randomization settings. Results show that the step-forward randomization with block urn design provides better performances over others. If the concern on the potential time delay is not serious and a central randomization system is available, the minimization method with an imbalance control threshold and a biased coin probability could be a better choice.

Non-invasive ventilation during exercise training for people with chronic obstructive pulmonary disease

BACKGROUND

Exercise training as a component of pulmonary rehabilitation improves health-related quality of life (HRQL) and exercise capacity in people with chronic obstructive pulmonary disease (COPD). However, some individuals may have difficulty performing exercise at an adequate intensity. Non-invasive ventilation (NIV) during exercise improves exercise capacity and dyspnoea during a single exercise session. Consequently, NIV during exercise training may allow individuals to exercise at a higher intensity, which could lead to greater improvement in exercise capacity, HRQL and physical activity.

OBJECTIVES

To determine whether NIV during exercise training (as part of pulmonary rehabilitation) affects exercise capacity, HRQL and physical activity in people with COPD compared with exercise training alone or exercise training with sham NIV.

SEARCH METHODS

We searched the following databases between January 1987 and November 2013 inclusive: The Cochrane Airways Group specialised register of trials, AMED, CENTRAL, CINAHL, EMBASE, LILACS, MEDLINE, PEDro, PsycINFO and PubMed. 

SELECTION CRITERIA

Randomised controlled trials that compared NIV during exercise training versus exercise training alone or exercise training with sham NIV in people with COPD were considered for inclusion in this review.

DATA COLLECTION AND ANALYSIS

Two review authors independently selected trials for inclusion in the review, extracted data and assessed risk of bias. Primary outcomes were exercise capacity, HRQL and physical activity; secondary outcomes were training intensity, physiological changes related to exercise training, dyspnoea, dropouts, adverse events and cost.

MAIN RESULTS

Six studies involving 126 participants who completed the study protocols were included. Most studies recruited participants with severe to very severe COPD (mean forced expiratory volume in one second (FEV1) ranged from 26% to 48% predicted). There was an increase in percentage change peak and endurance exercise capacity with NIV during training (mean difference in peak exercise capacity 17%, 95% confidence interval (CI) 7% to 27%, 60 participants, low-quality evidence; mean difference in endurance exercise capacity 59%, 95% CI 4% to 114%, 48 participants, low-quality evidence). However, there was no clear evidence of a difference between interventions for all other measures of exercise capacity. The results for HRQL assessed using the St George's Respiratory Questionnaire do not rule out an effect of NIV (total score mean 2.5 points, 95% CI -2.3 to 7.2, 48 participants, moderate-quality evidence). Physical activity was not assessed in any study. There was an increase in training intensity with NIV during training of 13% (95% CI 1% to 27%, 67 participants, moderate-quality evidence), and isoload lactate was lower with NIV (mean difference -0.97 mmol/L, 95% CI -1.58mmol/L to -0.36 mmol/L, 37 participants, moderate-quality evidence). The effect of NIV on dyspnoea or the number of dropouts between interventions was uncertain, although again results were imprecise. No adverse events and no information regarding cost were reported. Only one study blinded participants, whereas three studies used blinded assessors. Adequate allocation concealment was reported in four studies.

AUTHORS' CONCLUSIONS

The small number of included studies with small numbers of participants, as well as the high risk of bias within some of the included studies, limited our ability to draw strong evidence-based conclusions. Although NIV during lower limb exercise training may allow people with COPD to exercise at a higher training intensity and to achieve a greater physiological training effect compared with exercise training alone or exercise training with sham NIV, the effect on exercise capacity is unclear. Some evidence suggests that NIV during exercise training improves the percentage change in peak and endurance exercise capacity; however, these findings are not consistent across other measures of exercise capacity. There is no clear evidence that HRQL is better or worse with NIV during training. It is currently unknown whether the demonstrated benefits of NIV during exercise training are clinically worthwhile or cost-effective.

Influence of selection bias on the test decision. A simulation study

BACKGROUND

Selection bias arises in clinical trials by reason of selective assignment of patients to treatment groups. Even in randomized clinical trials with allocation concealment this phenomenon can occur if future assignments can be predicted due to knowledge of former allocations.

OBJECTIVES

Considering unmasked randomized clinical trials with allocation concealment the impact of selection bias on type I error rate under permuted block randomization is investigated. We aimed to extend the existing research into this topic by including practical assumptions concerning misclassification of patient characteristics to get an estimate of type I error close to clinical routine. To establish an upper bound for the type I error rate different biasing strategies of the investigator are compared first. In addition, the aspect of patient availability is considered.

METHODS

To evaluate the influence of selection bias on type I error rate under several practical situations, different block sizes, selection effects, biasing strategies and success rates of patient classification were simulated using SAS.

RESULTS

Type I error rate exceeds 5 percent significance level; it reaches values up to 21 percent. More cautious biasing strategies and misclassification of patient characteristics may diminish but cannot eliminate selection bias. The number of screened patients is about three times larger than the needed number for the trial.

CONCLUSIONS

Even in unmasked randomized clinical trials using permuted block randomization with allocation concealment the influence of selection bias must not be disregarded evaluating the test decision. It should be incorporated when designing and reporting a clinical trial.

The impact of selection bias on test decisions in randomized clinical trials

Selection bias affects the evaluation of clinical trials, for example, by elevating type I error rate. We investigated the effect of selection bias on type I error rate considering permuted block randomization. We also considered stratified randomization in general and for the special case of multicenter clinical trials, where we incorporated preferences of the investigator in our approach. Finally, the effect of underrunning is modeled, that is, where the randomization list exceeds the actual number of patients taking part in the trial. For all situations, we illustrate and discuss the impact of selection bias on type I error rate.

A simplified formula for quantification of the probability of deterministic assignments in permuted block randomization

Open label and single blinded randomized controlled clinical trials are vulnerable to selection bias when the next treatment assignment is predictable based on the randomization algorithm and the preceding assignment history. While treatment predictability is an issue for all constrained randomization algorithms, deterministic assignments are unique to permuted block randomization. Deterministic assignments may lead to treatment predictability with certainty and selection bias, which could inflate the type I error and hurts the validity of trial results. It is important to accurately evaluate the probability of deterministic assignments in permuted block randomization, so proper protection measures can be implemented. For trials with number of treatment arms T = 2 and a balance block size B = 2m, Matts and Lachin indicated that the probability of deterministic assignment is 1m+1. For more general situations, with T ≥ 2 and a block size B=∑j=1Tmj, Dupin-Spriet provided a formula, which can be written as 1B∑j=1T∑i=1mj∏k=1imj-k+1B-k+1. This formula involves extensive calculation in evaluation. In this paper, we simplified this formula to 1B∑j=1TmjB-mj+1 for general scenarios and 1B-m+1 for trials with a balanced allocation. Through mathematical induction we show the equivalence of the formulas. While the new formula is numerically equivalent to Dupin-Spriet's formula, the simple format not only is easier for evaluation, but also is clearer in describing the impact of parameters T and m(i) on the probability of deterministic assignments.

Balance algorithm for cluster randomized trials

Background

Within cluster randomized trials no algorithms exist to generate a full enumeration of a block randomization, balancing for covariates across treatment arms. Furthermore, often for practical reasons multiple blocks are required to fully randomize a study, which may not have been well balanced within blocks.

Results

We present a convenient and easy to use randomization tool to undertake allocation concealed block randomization. Our algorithm highlights allocations that minimize imbalance between treatment groups across multiple baseline covariates.

We demonstrate the algorithm using a cluster randomized trial in primary care (the PRE-EMPT Study) and show that the software incorporates a trade off between independent random allocations that were likely to be imbalanced, and predictable deterministic approaches that would minimise imbalance. We extend the methodology of single block randomization to allocate to multiple blocks conditioning on previous allocations.

Conclusion

The algorithm is included as Additional file 1 and we advocate its use for robust randomization within cluster randomized trials.

A critical appraisal of propensity-score matching in the medical literature between 1996 and 2003

Propensity-score methods are increasingly being used to reduce the impact of treatment-selection bias in the estimation of treatment effects using observational data. Commonly used propensity-score methods include covariate adjustment using the propensity score, stratification on the propensity score, and propensity-score matching. Empirical and theoretical research has demonstrated that matching on the propensity score eliminates a greater proportion of baseline differences between treated and untreated subjects than does stratification on the propensity score. However, the analysis of propensity-score-matched samples requires statistical methods appropriate for matched-pairs data. We critically evaluated 47 articles that were published between 1996 and 2003 in the medical literature and that employed propensity-score matching. We found that only two of the articles reported the balance of baseline characteristics between treated and untreated subjects in the matched sample and used correct statistical methods to assess the degree of imbalance. Thirteen (28 per cent) of the articles explicitly used statistical methods appropriate for the analysis of matched data when estimating the treatment effect and its statistical significance. Common errors included using the log-rank test to compare Kaplan-Meier survival curves in the matched sample, using Cox regression, logistic regression, chi-squared tests, t-tests, and Wilcoxon rank sum tests in the matched sample, thereby failing to account for the matched nature of the data. We provide guidelines for the analysis and reporting of studies that employ propensity-score matching.

The analysis of stratified 2 x 2 contingency tables

We consider the problem of testing for independence against the consistent superiority of one treatment over another when the response variable is binary and is compared across two treatments in each of several strata. Specifically, we consider the randomized clinical trial setting. A number of issues arise in this context. First, should tables be combined if there are small or zero margins? Second, should one assume a common odds ratio across strata? Third, if the odds ratios differ across strata, then how does the standard test (based on a common odds ratio) perform? Fourth, are there other analyzes that are more appropriate for handling a situation in which the odds ratios may differ across strata? In addressing these issues we find that the frequently used Cochran-Mantel-Haenszel test may have a poor power profile, despite being optimal when the odds ratios are common. We develop novel tests that are analogous to the Smirnov, modified Smirnov, convex hull, and adaptive tests that have been proposed for ordered categorical data.