Seven myths of randomisation in clinical trials

Evidenced-Based Prior for Estimating the Treatment Effect of Phase III Randomized Trials in Oncology

PURPOSE

The primary results of phase III oncology trials may be challenging to interpret, given that results are generally based on P value thresholds. The probability of whether a treatment is beneficial, although more intuitive, is not usually provided. Here, we developed and released a user-friendly tool that calculates the probability of treatment benefit using trial summary statistics.

METHODS

We curated 415 phase III randomized trials enrolling 338,600 patients published between 2004 and 2020. A phase III prior probability distribution for the treatment effect was developed on the basis of a three-component zero-mean mixture distribution of the observed z-scores. Using this prior, we computed the probability of clinically meaningful benefit (hazard ratio [HR] <0.8). The distribution of signal-to-noise ratios and power of phase III oncology trials were compared with that of 23,551 randomized trials from the Cochrane Database.

RESULTS

The signal-to-noise ratios of phase III oncology trials tended to be much larger than randomized trials from the Cochrane Database. Still, the median power of phase III oncology trials was only 49% (IQR, 14%-95%), and the power was <80% in 65% of trials. Using the phase III oncology-specific prior, only 53% of trials claiming superiority (114 of 216) had a ≥90% probability of clinically meaningful benefits. Conversely, the probability that the experimental arm was superior to the control arm (HR <1) exceeded 90% in 17% of trials interpreted as having no benefit (34 of 199).

CONCLUSION

By enabling computation of contextual probabilities for the treatment effect from summary statistics, our robust, highly practical tool, now posted on a user-friendly webpage, can aid the wider oncology community in the interpretation of phase III trials.

Increasing Power in Phase III Oncology Trials With Multivariable Regression: An Empirical Assessment of 535 Primary End Point Analyses

PURPOSE

A previous study demonstrated that power against the (unobserved) true effect for the primary end point (PEP) of most phase III oncology trials is low, suggesting an increased risk of false-negative findings in the field of late-phase oncology. Fitting models with prognostic covariates is a potential solution to improve power; however, the extent to which trials leverage this approach, and its impact on trial interpretation at scale, is unknown. To that end, we hypothesized that phase III trials using multivariable PEP analyses are more likely to demonstrate superiority versus trials with univariable analyses.

METHODS

PEP analyses were reviewed from trials registered on ClinicalTrials.gov. Adjusted odds ratios (aORs) were calculated by logistic regressions.

RESULTS

Of the 535 trials enrolling 454,824 patients, 69% (n = 368) used a multivariable PEP analysis. Trials with multivariable PEP analyses were more likely to demonstrate PEP superiority (57% [209 of 368] v 42% [70 of 167]; aOR, 1.78 [95% CI, 1.18 to 2.72]; P = .007). Among trials with a multivariable PEP model, 16 conditioned on covariates and 352 stratified by covariates. However, 108 (35%) of 312 trials with stratified analyses lost power by categorizing a continuous variable, which was especially common among immunotherapy trials (aOR, 2.45 [95% CI, 1.23 to 4.92]; P = .01).

CONCLUSION

Trials increasing power by fitting multivariable models were more likely to demonstrate PEP superiority than trials with unadjusted analysis. Underutilization of conditioning models and empirical power loss associated with covariate categorization required by stratification were identified as barriers to power gains. These findings underscore the opportunity to increase power in phase III trials with conventional methodology and improve patient access to effective novel therapies.

Demarcating scientific medicine

Scientific medicine and homeopathy are interesting case studies for the ongoing project of demarcating science from pseudoscience. Much of the demarcation literature formulates abstract criteria for demarcating science from pseudoscience generally. In service of a more localist approach to the demarcation problem, I reconstruct a specific demarcating difference, the like comparison criterion, invoked by nineteenth century adherents to an early model of scientific medicine. If it is to remain relevant today, I argue that the like comparison criterion must be updated in our current era of epidemiological, evidence-based medicine to recognize the importance of assessing study bias and mechanistic implausibility in contemporary medical science.

Applying EBM epistemology and the GRADE system to address practitioners' disagreements in medical malpractice allegations during COVID-19 pandemic

RATIONALE

The GRADE system of clinical recommendations has deontic implications and can discriminate between mandatory, prohibited, and merely permitted medical decisions.

AIMS AND OBJECTIVES

The recommendation categories of the GRADE framework map onto deontological imperatives that can lead to a better understanding and management of allegations of imprudence and appropriateness of treatments. Allegations made during the worst phase of COVID-19 pandemic are used as a case study for exploring the deontic implications of GRADE.

METHOD

Conceptual theoretical analysis, case study analysis, and argumentation in defence of hypotheses.

RESULTS

Strong GRADE recommendations for or against treatment are justified by high-quality evidence and can be construed as ethical obligations and prohibitions. In contrast, when evidence for benefit or harm is of lower quality, GRADE yields weak, discretionary recommendations. In such grey area cases, the absence of a duty to prescribe or refuse to prescribe a requested treatment is compatible with the privilege of considering unproven but possibly beneficial options in a private setting. This privilege, however, does not extend to healthcare policymakers, who have a duty to promote actions that serve the public and whose recommendations should not be guided by personal or idiosyncratic preferences or values.

CONCLUSION

If there is no prima facie evidence that a proposed treatment is harmful, doctors are not negligent in considering it in shared doctor-patient decision-making. But these clinical decisions under uncertainty do not transfer obligations to health authorities, who are not part of the decision-making process in clinical settings. The clinical decision-making process concerns particulars and is guided by contextual and specific reasons that do not fall within the scope of a general policy. Thus, in the context of a serious epidemic in which patients need and demand treatments, if the body of evidence is still changing and fragile, an attitude of tolerance and connivance may ensure a smoother transition to a more stable phase of progress, both in scientific and clinical medicine.

Bayesian Interim Analysis and Efficiency of Phase III Randomized Trials

IMPORTANCE

Improving the efficiency of interim assessments in phase III trials should reduce trial costs, hasten the approval of efficacious therapies, and mitigate patient exposure to disadvantageous randomizations.

OBJECTIVE

We hypothesized that in silico Bayesian early stopping rules improve the efficiency of phase III trials compared with the original frequentist analysis without compromising overall interpretation.

DESIGN

Cross-sectional analysis.

SETTING

230 randomized phase III oncology trials enrolling 184,752 participants.

PARTICIPANTS

Individual patient-level data were manually reconstructed from primary endpoint Kaplan-Meier curves.

INTERVENTIONS

Trial accruals were simulated 100 times per trial and leveraged published patient outcomes such that only the accrual dynamics, and not the patient outcomes, were randomly varied.

MAIN OUTCOMES AND MEASURES

Early stopping was triggered per simulation if interim analysis demonstrated ≥ 85% probability of minimum clinically important difference/3 for efficacy or futility. Trial-level early closure was defined by stopping frequencies ≥ 0.75.

RESULTS

A total of 12,451 simulations (54%) met early stopping criteria. Trial-level early stopping frequency was highly predictive of the published outcome (OR, 7.24; posterior probability of association, >99.99%; AUC, 0.91; P < 0.0001). Trial-level early closure was recommended for 82 trials (36%), including 62 trials (76%) which had performed frequentist interim analysis. Bayesian early stopping rules were 96% sensitive (95% CI, 91% to 98%) for detecting trials with a primary endpoint difference, and there was a high level of agreement in overall trial interpretation (Bayesian Cohen's κ, 0.95; 95% CrI, 0.92 to 0.99). However, Bayesian interim analysis was associated with >99.99% posterior probability of reducing patient enrollment requirements ( P < 0.0001), with an estimated cumulative enrollment reduction of 20,543 patients (11%; 89 patients averaged equally over all studied trials) and an estimated cumulative cost savings of 851 million USD (3.7 million USD averaged equally over all studied trials).

CONCLUSIONS AND RELEVANCE

Bayesian interim analyses may improve randomized trial efficiency by reducing enrollment requirements without compromising trial interpretation. Increased utilization of Bayesian interim analysis has the potential to reduce costs of late-phase trials, reduce patient exposures to ineffective therapies, and accelerate approvals of effective therapies.

KEY POINTS

Question: What are the effects of Bayesian early stopping rules on the efficiency of phase III randomized oncology trials?Findings: Individual-patient level outcomes were reconstructed for 184,752 patients from 230 trials. Compared with the original interim analysis strategy, in silico Bayesian interim analysis reduced patient enrollment requirements and preserved the original trial interpretation. Meaning: Bayesian interim analysis may improve the efficiency of conducting randomized trials, leading to reduced costs, reduced exposure of patients to disadvantageous treatments, and accelerated approval of efficacious therapies.

Use and reporting of inverse-probability-of-treatment weighting for multicategory treatments in medical research: a systematic review

OBJECTIVES

Causal inference methods for observational data represent an alternative to randomised controlled trials when they are not feasible or when real-world evidence is sought. Inverse-probability-of-treatment weighting (IPTW) is one of the most popular approaches to account for confounding in observational studies. In medical research, IPTW is mainly applied to estimate the causal effect of a binary treatment, even when the treatment has in fact multiple categories, despite the availability of IPTW estimators for multiple treatment categories. This raises questions about the appropriateness of the use of IPTW in this context. Therefore, we conducted a systematic review of medical publications reporting the use of IPTW in the presence of a multi-category treatment. Our objectives were to investigate the frequency of use and the implementation of these methods in practice, and to assess the quality of their reporting.

STUDY DESIGN AND SETTING

Using Pubmed, Embase and Web of Science, we screened 5660 articles and retained 106 articles in the final analysis that were from 17 different medical areas. This systematic review is registered on PROSPERO (CRD42022352669).

RESULTS

The number of treatment groups varied between 3 and 9, with a large majority of articles (90 [84.9%]) including 3 or 4 groups. The most commonly used method for estimating the weights was multinomial regression (51 [48.1%]) and generalized boosted models (48 [45.3%]). The covariates of the weight model were reported in 91 articles (85.9 %). Twenty-six articles (24.5 %) did not discuss the balance of covariates after weighting, and only 16 articles (15.1 %) referred to the assumptions needed to obtain correct inferences.

CONCLUSION

The results of this systematic review illustrate that medical publications scarcely use IPTW methods for more than two treatment categories. Among the publications that did, the quality of reporting was suboptimal, in particular in regard to the assumptions and model building. IPTW for multi-category treatments could be applied more broadly in medical research, and the application of the proposed guidelines in this context will help researchers to report their results and to ensure reproducibility of their research.

Covariate adjustment in Bayesian adaptive randomized controlled trials

In conventional randomized controlled trials, adjustment for baseline values of covariates known to be at least moderately associated with the outcome increases the power of the trial. Recent work has shown a particular benefit for more flexible frequentist designs, such as information adaptive and adaptive multi-arm designs. However, covariate adjustment has not been characterized within the more flexible Bayesian adaptive designs, despite their growing popularity. We focus on a subclass of these which allow for early stopping at an interim analysis given evidence of treatment superiority. We consider both collapsible and non-collapsible estimands and show how to obtain posterior samples of marginal estimands from adjusted analyses. We describe several estimands for three common outcome types. We perform a simulation study to assess the impact of covariate adjustment using a variety of adjustment models in several different scenarios. This is followed by a real-world application of the compared approaches to a COVID-19 trial with a binary endpoint. For all scenarios, it is shown that covariate adjustment increases power and the probability of stopping the trials early, and decreases the expected sample sizes as compared to unadjusted analyses.

Locoregional Therapies in Immunologically "Cold" Tumors: Opportunities and Clinical Trial Design Considerations

Immunotherapy has revolutionized cancer management, but many tumors, particularly immunologically "cold" tumors, remain resistant to the therapy. The combination of conventional systemic immunotherapies and locoregional interventional radiology approaches is being explored to transform these cold tumors into immunologically active "hot" ones. The present article uses the example of chromophobe renal cell carcinoma (ChRCC), a renal cell carcinoma subtype resistant to current systemic immunotherapies, to address practical and conceptual challenges that have prevented the activation of clinical trials specifically designed for this malignancy to date. The practical framework discussed herein can help overcome logistic and funding limitations and facilitate the development of biology-informed clinical trials tailored to specific rare diseases such as ChRCC.

Minimization in randomized clinical trials

In randomized trials, comparability of the treatment groups is ensured through allocation of treatments using a mechanism that involves some random element, thus controlling for confounding of the treatment effect. Completely random allocation ensures comparability between the treatment groups for all known and unknown prognostic factors. For a specific trial, however, imbalances in prognostic factors among the treatment groups may occur. Although accidental bias can be avoided in the presence of such imbalances by stratifying the analysis, most trialists, regulatory agencies, and other stakeholders prefer a balanced distribution of prognostic factors across the treatment groups. Some procedures attempt to achieve balance in baseline covariates, by stratifying the allocation for these covariates, or by dynamically adapting the allocation using covariate information during the trial (covariate-adaptive procedures). In this Tutorial, the performance of minimization, a popular covariate-adaptive procedure, is compared with two other commonly used procedures, completely random allocation and stratified blocked designs. Using individual patient data of 2 clinical trials (in advanced ovarian cancer and age-related macular degeneration), the procedures are compared in terms of operating characteristics (using asymptotic and randomization tests), predictability of treatment allocation, and achieved balance. Fifty actual trials of various sizes that applied minimization for treatment allocation are used to investigate the achieved balance. Implementation issues of minimization are described. Minimization procedures are useful in all trials but especially when (1) many major prognostic factors are known, (2) many centers of different sizes accrue patients, or (3) the trial sample size is moderate.

Prevalence and implications of significance testing for baseline covariate imbalance in randomised cancer clinical trials: The Table 1 Fallacy

BACKGROUND

The 'Table 1 Fallacy' refers to the unsound use of significance testing for comparing the distributions of baseline variables between randomised groups to draw erroneous conclusions about balance or imbalance. We performed a cross-sectional study of the Table 1 Fallacy in phase III oncology trials.

METHODS

From ClinicalTrials.gov, 1877 randomised trials were screened. Multivariable logistic regressions evaluated predictors of the Table 1 Fallacy.

RESULTS

A total of 765 randomised controlled trials involving 553,405 patients were analysed. The Table 1 Fallacy was observed in 25% of trials (188 of 765), with 3% of comparisons deemed significant (59 of 2353), approximating the typical 5% type I error assertion probability. Application of trial-level multiplicity corrections reduced the rate of significant findings to 0.3% (six of 2345 tests). Factors associated with lower odds of the Table 1 Fallacy included industry sponsorship (adjusted odds ratio [aOR] 0.29, 95% confidence interval [CI] 0.18-0.47; multiplicity-corrected P < 0.0001), larger trial size (≥795 versus <280 patients; aOR 0.32, 95% CI 0.19-0.53; multiplicity-corrected P = 0.0008), and publication in a European versus American journal (aOR 0.06, 95% CI 0.03-0.13; multiplicity-corrected P < 0.0001).

CONCLUSIONS

This study highlights the persistence of the Table 1 Fallacy in contemporary oncology randomised controlled trials, with one of every four trials testing for baseline differences after randomisation. Significance testing is a suboptimal method for identifying unsound randomisation procedures and may encourage misleading inferences. Journal-level enforcement is a possible strategy to help mitigate this fallacy.

Statistics, philosophy, and health: the SMAC 2021 webconference

SMAC 2021 was a webconference organized in June 2021. The aim of this conference was to bring together data scientists, (bio)statisticians, philosophers, and any person interested in the questions of causality and Bayesian statistics, ranging from technical to philosophical aspects. This webconference consisted of keynote speakers and contributed speakers, and closed with a round-table organized in an unusual fashion. Indeed, organisers asked world renowned scientists to prepare two videos: a short video presenting a question of interest to them and a longer one presenting their point of view on the question. The first video served as a "teaser" for the conference and the second were presented during the conference as an introduction to the round-table. These videos and this round-table generated original scientific insights and discussion worthy of being shared with the community which we do by means of this paper.

Impact of Allocation Concealment and Blinding in Trials Addressing Treatments for COVID-19: A Methods Study

We aimed to assess the impact of allocation concealment and blinding on the results of coronavirus disease 2019 (COVID-19) trials, using the World Health Organization COVID-19 database (to February 2022). We identified 488 randomized trials comparing drug therapeutics with placebo or standard care in patients with COVID-19. We performed random-effects meta-regressions comparing the results of trials with and without allocation concealment and blinding of health-care providers and patients. We found that, compared with trials with allocation concealment, trials without allocation concealment may estimate treatments to be more beneficial for mortality, mechanical ventilation, hospital admission, duration of hospitalization, and duration of mechanical ventilation, but results were imprecise. We did not find compelling evidence that, compared with trials with blinding, trials without blinding produce consistently different results for mortality, mechanical ventilation, and duration of hospitalization. We found that trials without blinding may estimate treatments to be more beneficial for hospitalizations and duration of mechanical ventilation. We did not find compelling evidence that COVID-19 trials in which health-care providers and patients are blinded produce different results from trials without blinding, but trials without allocation concealment estimate treatments to be more beneficial compared with trials with allocation concealment. Our study suggests that lack of blinding may not always bias results but that evidence users should remain skeptical of trials without allocation concealment.

Interpreting Randomized Controlled Trials

This article describes rationales and limitations for making inferences based on data from randomized controlled trials (RCTs). We argue that obtaining a representative random sample from a patient population is impossible for a clinical trial because patients are accrued sequentially over time and thus comprise a convenience sample, subject only to protocol entry criteria. Consequently, the trial's sample is unlikely to represent a definable patient population. We use causal diagrams to illustrate the difference between random allocation of interventions within a clinical trial sample and true simple or stratified random sampling, as executed in surveys. We argue that group-specific statistics, such as a median survival time estimate for a treatment arm in an RCT, have limited meaning as estimates of larger patient population parameters. In contrast, random allocation between interventions facilitates comparative causal inferences about between-treatment effects, such as hazard ratios or differences between probabilities of response. Comparative inferences also require the assumption of transportability from a clinical trial's convenience sample to a targeted patient population. We focus on the consequences and limitations of randomization procedures in order to clarify the distinctions between pairs of complementary concepts of fundamental importance to data science and RCT interpretation. These include internal and external validity, generalizability and transportability, uncertainty and variability, representativeness and inclusiveness, blocking and stratification, relevance and robustness, forward and reverse causal inference, intention to treat and per protocol analyses, and potential outcomes and counterfactuals.

Pair-switching rerandomization

Rerandomization discards assignments with covariates unbalanced in the treatment and control groups to improve estimation and inference efficiency. However, the acceptance-rejection sampling method used in rerandomization is computationally inefficient. As a result, it is time-consuming for rerandomization to draw numerous independent assignments, which are necessary for performing Fisher randomization tests and constructing randomization-based confidence intervals. To address this problem, we propose a pair-switching rerandomization (PSRR) method to draw balanced assignments efficiently. We obtain the unbiasedness and variance reduction of the difference-in-means estimator and show that the Fisher randomization tests are valid under PSRR. Moreover, we propose an exact approach to invert Fisher randomization tests to confidence intervals, which is faster than the existing methods. In addition, our method is applicable to both nonsequentially and sequentially randomized experiments. We conduct comprehensive simulation studies to compare the finite-sample performance of the proposed method with that of classical rerandomization. Simulation results indicate that PSRR leads to comparable power of Fisher randomization tests and is 3-23 times faster than classical rerandomization. Finally, we apply the PSRR method to analyze two clinical trial datasets, both of which demonstrate the advantages of our method.

A new method for analyzing clinical trials in depression based on individual propensity to respond to placebo estimated using artificial intelligence

One of the major reasons for trial failures in major depressive disorders (MDD) is the presence of unpredictable levels of placebo response as the individual baseline propensity to respond to placebo is not adequately controlled by the current randomization and statistical methodologies. The individual propensity to respond to any treatment or intervention assessed at baseline was considered as a major non-specific prognostic and confounding effect. The objective of this paper was to apply the propensity score methodology to control for potential imbalance at baseline in the propensity to respond to placebo in clinical trials in MDD. Individual propensity was estimated using artificial intelligence (AI) applied to observations collected in two pre-randomization occasions. Cases study are presented using data from two randomized, placebo-controlled trials to evaluate the efficacy of paroxetine in MDD. AI models were used to estimate the individual propensity probability to show a treatment non-specific placebo effect. The inverse of the estimated probability was used as weight in the mixed-effects analysis to assess treatment effect. The comparison of the results obtained with and without propensity weight indicated that the weighted analysis provided an estimate of treatment effect and effect size significantly larger than the conventional analysis.

Evaluation of randomized controlled trials: a primer and tutorial for mental health researchers

BACKGROUND

Considered one of the highest levels of evidence, results of randomized controlled trials (RCTs) remain an essential building block in mental health research. They are frequently used to confirm that an intervention "works" and to guide treatment decisions. Given their importance in the field, it is concerning that the quality of many RCT evaluations in mental health research remains poor. Common errors range from inadequate missing data handling and inappropriate analyses (e.g., baseline randomization tests or analyses of within-group changes) to unduly interpretations of trial results and insufficient reporting. These deficiencies pose a threat to the robustness of mental health research and its impact on patient care. Many of these issues may be avoided in the future if mental health researchers are provided with a better understanding of what constitutes a high-quality RCT evaluation.

METHODS

In this primer article, we give an introduction to core concepts and caveats of clinical trial evaluations in mental health research. We also show how to implement current best practices using open-source statistical software.

RESULTS

Drawing on Rubin's potential outcome framework, we describe that RCTs put us in a privileged position to study causality by ensuring that the potential outcomes of the randomized groups become exchangeable. We discuss how missing data can threaten the validity of our results if dropouts systematically differ from non-dropouts, introduce trial estimands as a way to co-align analyses with the goals of the evaluation, and explain how to set up an appropriate analysis model to test the treatment effect at one or several assessment points. A novice-friendly tutorial is provided alongside this primer. It lays out concepts in greater detail and showcases how to implement techniques using the statistical software R, based on a real-world RCT dataset.

DISCUSSION

Many problems of RCTs already arise at the design stage, and we examine some avoidable and unavoidable "weak spots" of this design in mental health research. For instance, we discuss how lack of prospective registration can give way to issues like outcome switching and selective reporting, how allegiance biases can inflate effect estimates, review recommendations and challenges in blinding patients in mental health RCTs, and describe problems arising from underpowered trials. Lastly, we discuss why not all randomized trials necessarily have a limited external validity and examine how RCTs relate to ongoing efforts to personalize mental health care.

Randomized controlled trials and alternative study designs in surgical oncology

Surgery is central to the cure of most solid cancers and an integral part of modern multimodal cancer management for early and advanced stage cancers. Decisions made by surgeons and multidisciplinary team members are based on best available knowledge for the defined clinical situation at hand. While surgery is both an art and a science, good decision-making requires data that are robust, valid, representative and, applicable to most if not all patients with a specific cancer. Such data largely comes from clinical observations and registries, and more preferably from trials conducted with the specific purpose of arriving at new answers. As part of the ESSO core curriculum development an increased focus has been put on the need to enhance research literacy among surgical candidates. As an expansion of the curriculum catalogue list and to enhance the educational value, we here present a set of principles and emerging concepts which applies to surgical oncologist for reading, understanding, planning and contributing to future surgeon-led cancer trials.

Evaluation of randomised controlled trials published in Indian specialty dental journals for statistical testing of baseline differences: A meta-epidemiological study

Background

In randomised controlled trials (RCTs), the application of a test of significance to compare the baseline differences between the intervention groups is a common practice, though it has been condemned by many researchers.

Objective

This study aimed to assess the proportion of RCTs on human participants comparing the baseline differences between intervention groups using the test of significance in nine dental specialty journals published in India and to estimate the proportion of studies reporting baseline demographic and clinical characteristics in a table.

Materials and Methods

RCTs published in nine dental journals published by dental specialty associations of India were screened. A literature search was limited to the time duration of five years from 2017 to 2021.

Results

The authors analysed 326 RCTs. Of 326 RCTs published, 237 RCTs (72.7%) did not report the baseline demographic and clinical characteristics table. Tests of significance were applied to compare baseline differences between the intervention arms in 148 (45.4%) RCTs published.

Conclusions

Although criticised by the Consolidated Standards of Reporting Trials (CONSORT) statement, the majority of the trials published in dental specialty journals failed to avoid comparison of baseline differences with significance test and failed to report baseline characteristic table.

Minimisation for the design of parallel cluster-randomised trials: An evaluation of balance in cluster-level covariates and numbers of clusters allocated to each arm

BACKGROUND

Cluster-randomised trials often use some form of restricted randomisation, such as stratified- or covariate-constrained randomisation. Minimisation has the potential to balance on more covariates than blocked stratification and can be implemented sequentially unlike covariate-constrained randomisation. Yet, unlike stratification, minimisation has no inbuilt guard to maintain close to a 1:1 allocation. A departure from a 1:1 allocation can be unappealing in a setting with a small number of allocation units such as cluster randomisation which typically include about 30 clusters.

METHODS

Using simulation (10,000 per scenario), we evaluate the performance of a range of minimisation procedures on the likelihood of a 1:1 allocation of clusters (10-80 clusters) to treatment arms, along with its performance on covariate imbalance. The range of minimisation procedures includes varying: the proportion of clusters allocated to the least imbalanced arm (known as the stochastic element) - between 0.7 and 1, percentage of first clusters allocated completely at random (known as the bed-in period) - between 0% and 20% and adding 'number of clusters allocated to each arm' as a covariate in the minimisation algorithm. We additionally include a comparison of stratifying and then minimising within key strata (such as country within a multi country cluster trial) as a potential aid to increasing balance.

RESULTS

Minimisation is unlikely to result in an exact 1:1 allocation unless the stochastic element is set higher than 0.9. For example, with 20 clusters, 2 binary covariates and setting the stochastic element to 0.7: only 41% of the possible randomisations over the 10,000 simulations achieved a 1:1 allocation. While typical sizes of imbalance were small (a difference of two clusters per arm), allocations as extreme as of 10:10 were observed. Adding the 'number of clusters' into the minimisation algorithm reduces this risk slightly, but covariate imbalance increases slightly. Stratifying and then minimising within key strata improve balance within strata but increase imbalance across all clusters, both on the number of clusters and covariate imbalance.

CONCLUSION

In cluster trials, where there are typically about 30 allocation units, when using minimisation, unless the stochastic element is set very high, there is a high risk of not achieving a 1:1 allocation, and a small but nonetheless real risk of an extreme departure from a 1:1 allocation. Stratification with minimisation within key strata (such as country) improves the balance within strata although compromises overall balance.

A matching procedure for sequential experiments that iteratively learns which covariates improve power

We propose a dynamic allocation procedure that increases power and efficiency when measuring an average treatment effect in sequential randomized trials exploiting some subjects' previous assessed responses. Subjects arrive sequentially and are either randomized or paired to a previously randomized subject and administered the alternate treatment. The pairing is made via a dynamic matching criterion that iteratively learns which specific covariates are important to the response. We develop estimators for the average treatment effect as well as an exact test. We illustrate our method's increase in efficiency and power over other allocation procedures in both simulated scenarios and a clinical trial dataset. An R package "SeqExpMatch" for use by practitioners is available on CRAN.

A comparison of covariate adjustment approaches under model misspecification in individually randomized trials

Adjustment for baseline covariates in randomized trials has been shown to lead to gains in power and can protect against chance imbalances in covariates. For continuous covariates, there is a risk that the the form of the relationship between the covariate and outcome is misspecified when taking an adjusted approach. Using a simulation study focusing on individually randomized trials with small sample sizes, we explore whether a range of adjustment methods are robust to misspecification, either in the covariate-outcome relationship or through an omitted covariate-treatment interaction. Specifically, we aim to identify potential settings where G-computation, inverse probability of treatment weighting (IPTW), augmented inverse probability of treatment weighting (AIPTW) and targeted maximum likelihood estimation (TMLE) offer improvement over the commonly used analysis of covariance (ANCOVA). Our simulations show that all adjustment methods are generally robust to model misspecification if adjusting for a few covariates, sample size is 100 or larger, and there are no covariate-treatment interactions. When there is a non-linear interaction of treatment with a skewed covariate and sample size is small, all adjustment methods can suffer from bias; however, methods that allow for interactions (such as G-computation with interaction and IPTW) show improved results compared to ANCOVA. When there are a high number of covariates to adjust for, ANCOVA retains good properties while other methods suffer from under- or over-coverage. An outstanding issue for G-computation, IPTW and AIPTW in small samples is that standard errors are underestimated; they should be used with caution without the availability of small-sample corrections, development of which is needed. These findings are relevant for covariate adjustment in interim analyses of larger trials.

Less is More? First Impressions From COSMIC-313

The COSMIC-313 phase 3 randomized controlled trial tested the triplet combination of cabozantinib with nivolumab and ipilimumab in comparison with nivolumab plus ipilimumab control as fist-line systemic therapy in metastatic clear cell renal cell carcinoma. The first results presented at the 2022 European Society of Medical Oncology Congress are a milestone for the renal cell carcinoma field because they signal the advent of triplet combinations as potential treatment options for our patients. The present commentary highlights some considerations and potential next steps based on these first impressions.

A systematic review of randomisation method use in RCTs and association of trial design characteristics with method selection

BACKGROUND

When conducting a randomised controlled trial, there exist many different methods to allocate participants, and a vast array of evidence-based opinions on which methods are the most effective at doing this, leading to differing use of these methods. There is also evidence that study characteristics affect the performance of these methods, but it is unknown whether the study design affects researchers' decision when choosing a method.

METHODS

We conducted a review of papers published in five journals in 2019 to assess which randomisation methods are most commonly being used, as well as identifying which aspects of study design, if any, are associated with the choice of randomisation method. Randomisation methodology use was compared with a similar review conducted in 2014.

RESULTS

The most used randomisation method in this review is block stratification used in 162/330 trials. A combination of simple, randomisation, block randomisation, stratification and minimisation make up 318/330 trials, with only a small number of more novel methods being used, although this number has increased marginally since 2014. More complex methods such as stratification and minimisation seem to be used in larger multicentre studies.

CONCLUSIONS

Within this review, most methods used can be classified using a combination of simple, block stratification and minimisation, suggesting that there is not much if any increase in the uptake of newer more novel methods. There seems to be a noticeable polarisation of method use, with an increase in the use of simple methods, but an increase in the complexity of more complex methods, with greater numbers of variables included in the analysis, and a greater number of strata.

Survival and stability of strategic mini-implants with immediate or delayed loading under removable partial dentures: a 3-year randomized controlled clinical trial

Objectives

Stability values of mini-implants (MIs) are ambiguous. Survival data for MIs as supplementary abutments in reduced dentitions are not available. The aim of this explorative research was to estimate the 3-year stability and survival of strategic MIs after immediate and delayed loading by existing removable partial dentures (RPDs).

Material and methods

In a university and three dental practices, patients with unfavorable tooth distributions received supplementary MIs with diameters of 1.8, 2.1, and 2.4 mm. The participants were randomly allocated to group A (if the insertion torque ≥ 35 Ncm: immediate loading by housings; otherwise, immediate loading by RPD soft relining was performed) or delayed loading group B. Periotest values (PTVs) and resonance frequency analysis (RFA) values were longitudinally compared using mixed models.

Results

A total of 112 maxillary and 120 mandibular MIs were placed under 79 RPDs (31 maxillae). The 1st and 3rd quartile of the PTVs ranged between 1.7 and 7.8, and the RFA values ranged between 30 and 46 with nonrelevant group differences. The 3-year survival rates were 92% in group A versus 95% in group B and 99% in the mandible (one failure) versus 87% in the maxilla (eleven failures among four participants).

Conclusions

Within the limitations of explorative analyses, there were no relevant differences between immediate and delayed loading regarding survival or stability of strategic MIs.

Clinical relevance

The stability values for MIs are lower than for conventional implants. The MI failure rate in the maxilla is higher than in the mandible with cluster failure participants.

Clinical trial registration

German Clinical Trials Register (Deutsches Register Klinischer Studien, DRKS-ID: DRKS00007589, www.germanctr.de), January 15, 2015.

A systematic review of whether COVID-19 randomized controlled trials reported on demographic and clinical characteristics

PURPOSE

We aim to assess the reporting of key patient-level demographic and clinical characteristics among COVID-19 related randomized controlled trials (RCTs).

METHODS

We queried English-language articles from PubMed, Web of Science, clinicaltrials.gov, and the CDC library of gray literature databases using keywords of "coronavirus," "covid," "clinical trial" and "randomized controlled trial" from January 2020 to June 2021. From the search, we conducted an initial review to rule-out duplicate entries, identify those that met inclusion criteria (i.e., had results), and exclude those that did not meet the definition of an RCT. Lastly, we abstracted the demographic and clinical characteristics reported on within each RCT.

RESULTS

From the initial 43 627 manuscripts, our final eligible manuscripts consisted of 149 RCTs described in 137 articles. Most of the RCTs (113/149) studied potential treatments, while fewer studied vaccines (29), prophylaxis strategies (5), and interventions to prevent transmission among those infected (2). Study populations ranged from 10 to 38 206 participants (median = 100, IQR: 60-300). All 149 RCTs reported on age, 147 on sex, 50 on race, and 110 on the prevalence of at least one comorbidity. No RCTs reported on income, urban versus rural residence, or other indicators of socioeconomic status (SES).

CONCLUSIONS

Limited reporting on race and other markers of SES make it difficult to draw conclusions about specific external target populations without making strong assumptions that treatment effects are homogenous. These findings highlight the need for more robust reporting on the clinical and demographic profiles of patients enrolled in COVID-19 related RCTs.

Baseline imbalance and heterogeneity are present in meta-analyses of randomized clinical trials examining the effects of exercise and medicines for blood pressure management

Randomized clinical trials attempt to reduce bias and create similar groups at baseline to infer causal effects. In meta-analyses, baseline imbalance may threaten the validity of the treatment effects. This meta-epidemiological study examined baseline imbalance in comparisons of exercise and antihypertensive medicines. Baseline data for systolic blood pressure, diastolic blood pressure, and age were extracted from a network meta-analysis of 391 randomized trials comparing exercise types and antihypertensive medicines. Fixed-effect meta-analyses were used to determine the presence of baseline imbalance and/or inconsistency. Meta-regression analyses were conducted on sample size, the risk of bias for allocation concealment, and whether data for all randomized participants were presented at baseline. In one exercise comparison, the resistance group was 0.3 years younger than the control group (95% confidence interval 0.6 to 0.1). Substantial inconsistency was observed in other exercise comparisons. Less data were available for medicines, but there were no occurrences of baseline imbalance and only a few instances of inconsistency. Several moderator analyses identified significant associations. We identified baseline imbalance as well as substantial inconsistency in exercise comparisons. Researchers should consider conducting meta-analyses of key prognostic variables at baseline to ensure balance across trials.

Study Types in Orthopaedics Research: Is My Study Design Appropriate for the Research Question?

When performing orthopaedic clinical research, alternative study designs can be more appropriate depending on the research question, availability of data, and feasibility. The most common observational study designs in total joint arthroplasty research are cohort and cross-sectional studies. This article describes methodological considerations for different study designs with examples from the total joint arthroplasty literature. We highlight the advantages and feasibility of experimental and observational study designs using real-world examples. We illustrate how to avoid common mistakes, such as incorrect labeling of matched cohort studies as case-control studies. We further guide investigators through a step-by-step design of a case-control study. We conclude with considerations when choosing between alternative study designs. Please visit the followinghttps://youtu.be/Zvce61cMYi8for videos that explain the highlights of the article in practical terms.

Nonmyopic and pseudo-nonmyopic approaches to optimal sequential design in the presence of covariates

In sequential experiments, subjects become available for the study over a period of time, and covariates are often measured at the time of arrival. We consider the setting where the sample size is fixed but covariate values are unknown until subjects enrol. Given a model for the outcome, a sequential optimal design approach can be used to allocate treatments to minimize the variance of the estimator of the treatment effect. We extend existing optimal design methodology so it can be used within a nonmyopic framework, where treatment allocation for the current subject depends not only on the treatments and covariates of the subjects already enrolled in the study, but also the impact of possible future treatment assignments within a specified horizon. The nonmyopic approach requires recursive formulae and suffers from the curse of dimensionality. We propose a pseudo-nonmyopic approach which has a similar aim to the nonmyopic approach, but does not involve recursion and instead relies on simulating trajectories of future possible decisions. Our simulation studies show that, for the simple case of a logistic regression with a single binary covariate and a binary treatment, and a more realistic case with four binary covariates, binary treatment and treatment-covariate interactions, the nonmyopic and pseudo-nonmyopic approaches provide no competitive advantage over the myopic approach, both in terms of the size of the estimated treatment effect and also the efficiency of the designs. Results are robust to the size of the horizon used in the nonmyopic approach, and the number of simulated trajectories used in the pseudo-nonmyopic approach.

Diets differing in carbohydrate cellularity and amount similarly reduced visceral fat in people with obesity - a randomized controlled trial (CARBFUNC)

BACKGROUND & AIMS

Visceral adipose tissue (VAT) volume is associated with common lifestyle diseases. Dietary quality, including food matrix and degree of carbohydrate cellularity, as well as the carbohydrate/fat ratio, may influence VAT volume. We aimed to determine the effects of isocaloric diets differing in either "cellularity", a novel marker of dietary carbohydrate quality, or carbohydrate amount on visceral fat volume and anthropometric measures in adults with obesity.

METHODS

In a randomized controlled trial of 193 people with obesity/central adiposity, we compared changes in VAT volume after 6 and 12 months, measured by abdominal computed tomography, on three isocaloric eating patterns based on "acellular" carbohydrate sources (e.g., flour-based whole-grain products; comparator arm), "cellular" carbohydrate sources (minimally processed foods with intact cellular structures such as fruits, potatoes/tubers, and rice), or low-carbohydrate high-fat (LCHF) principles. Outcomes were compared by an intention-to-treat (ITT) analysis using constrained linear mixed-effects modelling (cLMM) providing baseline-adjusted change scores and proper missing data handling without imputation.

RESULTS

78 and 57 participants completed 6 and 12 months, respectively, with similar intakes of energy (females: 1820-2060 kcal, males: 2480-2550 kcal) and protein (16-17 energy percent, E%) throughout the intervention, and only modest reductions in energy from baseline. Reported dietary intakes were 42-44, 41-42, and 11-15 E% carbohydrate and 36-38, 37-38, and 66-70 E% fat in the acellular, cellular and LCHF groups, respectively. There were no significant between-group differences in VAT volume after 6 months (cellular vs. acellular [95% CI]: -55 cm³ [-545, 436]; LCHF vs. acellular [95% CI]: -225 cm³ [-703, 253]) or after 12 months (cellular vs. acellular [95% CI]: -122 cm³ [-757, 514]; LCHF vs. acellular [95% CI]: -317 cm³ [-943, 309]). VAT volume decreased significantly within all groups by 14-18% and 12-17% after 6 and 12 months, respectively. Waist circumference was reduced to a significantly greater degree in the LCHF vs. acellular group at 6 months (LCHF vs. acellular [95% CI]: -2.78 cm [-5.54, -0.017]).

CONCLUSIONS

Despite modest energy restriction, the three isocaloric eating patterns, differing in carbohydrate cellularity and amount, decreased visceral fat volume significantly and to a similar clinically relevant degree.

CLINICAL TRIALS IDENTIFIER

NCT03401970. https://clinicaltrials.gov/ct2/show/NCT03401970.

MA-cont:pre/post effect size: An interactive tool for the meta-analysis of continuous outcomes using R Shiny

Meta-analysis is a widely used methodology to combine evidence from different sources examining a common research phenomenon, to obtain a quantitative summary of the studied phenomenon. In the medical field, multiple studies investigate the effectiveness of new treatments and meta-analysis is largely performed to generate the summary (average) treatment effect. In the meta-analysis of aggregate continuous outcomes measured in a pretest-posttest design using differences in means as the effect measure, a plethora of methods exist: analysis of final (follow-up) scores, analysis of change scores and analysis of covariance. Specialised and general-purpose statistical software is used to apply the various methods, yet, often the choice among them depends on data availability and statistical affinity. We present a new web-based tool, MA-cont:pre/post effect size, to conduct meta-analysis of continuous data assessed pre- and post-treatment using the aforementioned approaches on aggregate data and a more flexible approach of generating and analysing pseudo individual participant data. The interactive web environment, available by R Shiny, is used to create this free-to-use statistical tool, requiring no programming skills by the users. A basic statistical understanding of the methods running in the background is a prerequisite and we encourage the users to seek advice from technical experts when necessary.

Inferential Pluralism in Causal Reasoning from Randomized Experiments

Causal pluralism can be defended not only in respect to causal concepts and methodological guidelines, but also at the finer-grained level of causal inference from a particular source of evidence for causation. An argument for this last variety of pluralism is made based on an analysis of causal inference from randomized experiments (RCTs). Here, the causal interpretation of a statistically significant association can be established via multiple paths of reasoning, each relying on different assumptions and providing distinct elements of information in favour of a causal interpretation.

The design and analysis of vaccine trials for COVID-19 for the purpose of estimating efficacy

After a preliminary explanation as to how I came to know Andy Grieve and some remarks about his career and mine and how they have intersected, I consider the design and analysis of trials of vaccines for COVID‐19 for the purpose of estimating efficacy. Five large trials, run by the sponsors Pfizer/BioNTech, AstraZeneca/Oxford University, Moderna, Novavax and J&J Janssen are considered briefly. Frequentist approaches to analysis were used for four of the trials but Pfizer/BioNTech nominated a Bayesian approach. The design and analysis of this trial is considered in some detail, in particular as regards the choice of prior distribution. I conclude by drawing some general lessons.

Planning a method for covariate adjustment in individually randomised trials: a practical guide

BACKGROUND

It has long been advised to account for baseline covariates in the analysis of confirmatory randomised trials, with the main statistical justifications being that this increases power and, when a randomisation scheme balanced covariates, permits a valid estimate of experimental error. There are various methods available to account for covariates but it is not clear how to choose among them.

METHODS

Taking the perspective of writing a statistical analysis plan, we consider how to choose between the three most promising broad approaches: direct adjustment, standardisation and inverse-probability-of-treatment weighting.

RESULTS

The three approaches are similar in being asymptotically efficient, in losing efficiency with mis-specified covariate functions and in handling designed balance. If a marginal estimand is targeted (for example, a risk difference or survival difference), then direct adjustment should be avoided because it involves fitting non-standard models that are subject to convergence issues. Convergence is most likely with IPTW. Robust standard errors used by IPTW are anti-conservative at small sample sizes. All approaches can use similar methods to handle missing covariate data. With missing outcome data, each method has its own way to estimate a treatment effect in the all-randomised population. We illustrate some issues in a reanalysis of GetTested, a randomised trial designed to assess the effectiveness of an electonic sexually transmitted infection testing and results service.

CONCLUSIONS

No single approach is always best: the choice will depend on the trial context. We encourage trialists to consider all three methods more routinely.

Mind the gap: covariate constrained randomisation can protect against substantial power loss in parallel cluster randomised trials

Background

Cluster randomised trials often randomise a small number of units, putting them at risk of poor balance of covariates across treatment arms. Covariate constrained randomisation aims to reduce this risk by removing the worst balanced allocations from consideration. This is known to provide only a small gain in power over that averaged under simple randomisation and is likely influenced by the number and prognostic effect of the covariates.

We investigated the performance of covariate constrained randomisation in comparison to the worst balanced allocations, and considered the impact on the power of the prognostic effect and number of covariates adjusted for in the analysis.

Methods

Using simulation, we examined the Monte Carlo type I error rate and power of cross-sectional, two-arm parallel cluster-randomised trials with a continuous outcome and four binary cluster-level covariates, using either simple or covariate constrained randomisation. Data were analysed using a small sample corrected linear mixed-effects model, adjusted for some or all of the binary covariates. We varied the number of clusters, intra-cluster correlation, number and prognostic effect of covariates balanced in the randomisation and adjusted in the analysis, and the size of the candidate set from which the allocation was selected. For each scenario, 20,000 simulations were conducted.

Results

When compared to the worst balanced allocations, covariate constrained randomisation with an adjusted analysis provided gains in power of up to 20 percentage points. Even with analysis-based adjustment for those covariates balanced in the randomisation, the type I error rate was not maintained when the intracluster correlation is very small (0.001). Generally, greater power was achieved when more prognostic covariates are restricted in the randomisation and as the size of the candidate set decreases. However, adjustment for weakly prognostic covariates lead to a loss in power of up to 20 percentage points.

Conclusions

When compared to the worst balanced allocations, covariate constrained randomisation provides moderate to substantial improvements in power. However, the prognostic effect of the covariates should be carefully considered when selecting them for inclusion in the randomisation.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-022-01588-8.

Improvement of Oxidative Stress in Older Women Is Dependent on Resistance Training Volume: Active Aging Longitudinal Study

ABSTRACT

Cunha, PM, Ribeiro, AS, Padilha, C, Nunes, JP, Schoenfeld, BJ, Cyrino, LT, Tomeleri, CM, Nascimento, MA, Antunes, M, Fernandes, RR, Barbosa, DS, Venturini, D, Burini, RC, Sardinha, LB, and Cyrino, ES. Improvement of oxidative stress in older women is dependent on resistance training volume: Active aging longitudinal study. J Strength Cond Res 36(4): 1141-1146, 2022-The purpose of the present study was to investigate the effects of resistance training (RT) performed with a higher versus lower training volume on oxidative stress (OS) biomarkers in older women. Thirty-eight older women (≥60 years) were randomly assigned to 1 of 2 groups: a group that performed 1 set per exercise (low volume [LV], n = 18) or 3 sets per exercise (high volume [HV], n = 20). The whole-body RT consisted of a 12-week RT program involving 8 exercises performed with sets of 10-15 repetitions maximum, 3 days per week. Advanced oxidation protein products (AOPP), total radical-trapping antioxidant parameter (TRAP), and ferrous oxidation-xylenol orange (FOX) were used as OS biomarkers. The composite Z-score of the percentage changes from pre- to posttraining of OS biomarkers according to groups was calculated. A significant main effect of time (p < 0.05) was found for AOPP (LV = -7.3% vs. HV = -12.2%) and TRAP (LV = +1.5% vs. HV = +15.5%) concentrations, without a statistical difference between the groups (p > 0.05). A significant group vs. time interaction (p < 0.001) was revealed for FOX (LV = +6.4% vs. HV = -8.9%). The overall analysis indicated higher positive changes for HV than LV (composed Z-score: HV = 0.41 ± 1.22 vs. LV = -0.37 ± 1.03; p < 0.05). Our results suggest that a greater volume of RT seems to promote superior improvements on OS biomarkers in older women.

Patient-reported outcomes as hospital performance measures: the challenge of confounding and how to handle it

It is highly appealing to use patient-reported outcomes (PROs) as hospital performance measures; however, so far, the attention to key methodological issues has been limited. One of the most critical challenges when comparing PRO-based performance measures across providers is to rule out confounding. In this paper, we explain confounding and why it matters when comparing across providers. Using examples from studies, we present potential strategies for dealing with confounding when using PRO data at an aggregated level. We aim to give clinicians an overview of how confounding can be addressed in both the design stage (restriction, matching, self-controlled design and propensity score) and the analysis stage (stratification, standardization and multivariable adjustment, including multilevel analysis) of a study. We also briefly discuss strategies for confounding control when data on important confounders are missing or unavailable.

Errors in the implementation, analysis, and reporting of randomization within obesity and nutrition research: a guide to their avoidance

Randomization is an important tool used to establish causal inferences in studies designed to further our understanding of questions related to obesity and nutrition. To take advantage of the inferences afforded by randomization, scientific standards must be upheld during the planning, execution, analysis, and reporting of such studies. We discuss ten errors in randomized experiments from real-world examples from the literature and outline best practices for their avoidance. These ten errors include: representing nonrandom allocation as random, failing to adequately conceal allocation, not accounting for changing allocation ratios, replacing subjects in nonrandom ways, failing to account for non-independence, drawing inferences by comparing statistical significance from within-group comparisons instead of between-groups, pooling data and breaking the randomized design, failing to account for missing data, failing to report sufficient information to understand study methods, and failing to frame the causal question as testing the randomized assignment per se. We hope that these examples will aid researchers, reviewers, journal editors, and other readers to endeavor to a high standard of scientific rigor in randomized experiments within obesity and nutrition research.

Impervious to Randomness: Confounding and Selection Biases in Randomized Clinical Trials

The random allocation of therapies in randomized clinical trials is a powerful tool that removes all confounding biases that can affect treatment assignment. However, confounders influencing mediators of the treatment effect are unaffected by randomization and should be considered during trial design and statistical modeling.Examples of such mediators include biomarkers predictive of response to targeted therapies in oncology. Patient selection for such biomarkers is prudent in clinical trials. Conversely, prognostic information on outcome heterogeneity can be derived from observational datasets that include more representative populations. The fusion of experimental and observational data can then allow patient-specific inferences.

Reacting to prognostic covariate imbalance in randomised controlled trials

Clinical trialists may regard an observed imbalance on a prognostic covariate as sufficiently troubling to warrant action.

OBJECTIVE

To elucidate the issues associated with selecting, and switching between, an unadjusted versus an adjusted analysis in response to an observed covariate imbalance.

STUDY DESIGN AND SETTING

Simulation study performed under the null hypothesis of no treatment effect using data from a large secondary prevention trial of statin therapy. The operating characteristics of three reaction strategies to baseline imbalances observed post-hoc were assessed.

RESULTS

Unadjusted analyses produced valid p-values irrespective of chance imbalance on a prognostic covariate. Switching to an adjusted analysis introduced no bias when the decision was made without knowledge of the direction of the imbalance. When the decision was based on the direction of the imbalance, the risk of incorrectly declaring the experimental treatment superior was inflated (by up to 48% in the scenarios investigated).

CONCLUSION

Overreaction to baseline imbalances observed post-hoc is unwarranted and we support adherence to the ICH guideline recommendations on the use of covariates. A legitimate case for switching to an adjusted analysis prior to finalisation of the statistical analysis plan (SAP) could nevertheless be potentially made provided that the direction of an observed covariate imbalance is unknown. Investigators should avoid reviewing the distribution of baseline characteristics across randomised groups in an unblinded fashion, for open-label and blinded studies alike, prior to finalisation of the SAP.

WHAT IS NEW

ICH guidelines on adjustment for covariates in RCT analyses appropriately advise against overreaction to baseline imbalances observed post-hoc. CONSORT reporting guidelines nevertheless place an emphasis on comparability of baseline characteristics across randomised groups. We demonstrate through a series of simulation studies why the ICH guidance is sound, but that a switch to an adjusted analysis in reaction to an observed prognostic covariate imbalance could legitimately be made provided that, when reaching the decision, treatment allocation is masked, and the direction of the imbalance is unknown. Trialists should therefore consider preserving the masking of actual treatment assignment when assessing the distribution of baseline characteristics across randomised groups.

A roadmap to using randomization in clinical trials

BACKGROUND

Randomization is the foundation of any clinical trial involving treatment comparison. It helps mitigate selection bias, promotes similarity of treatment groups with respect to important known and unknown confounders, and contributes to the validity of statistical tests. Various restricted randomization procedures with different probabilistic structures and different statistical properties are available. The goal of this paper is to present a systematic roadmap for the choice and application of a restricted randomization procedure in a clinical trial.

METHODS

We survey available restricted randomization procedures for sequential allocation of subjects in a randomized, comparative, parallel group clinical trial with equal (1:1) allocation. We explore statistical properties of these procedures, including balance/randomness tradeoff, type I error rate and power. We perform head-to-head comparisons of different procedures through simulation under various experimental scenarios, including cases when common model assumptions are violated. We also provide some real-life clinical trial examples to illustrate the thinking process for selecting a randomization procedure for implementation in practice.

RESULTS

Restricted randomization procedures targeting 1:1 allocation vary in the degree of balance/randomness they induce, and more importantly, they vary in terms of validity and efficiency of statistical inference when common model assumptions are violated (e.g. when outcomes are affected by a linear time trend; measurement error distribution is misspecified; or selection bias is introduced in the experiment). Some procedures are more robust than others. Covariate-adjusted analysis may be essential to ensure validity of the results. Special considerations are required when selecting a randomization procedure for a clinical trial with very small sample size.

CONCLUSIONS

The choice of randomization design, data analytic technique (parametric or nonparametric), and analysis strategy (randomization-based or population model-based) are all very important considerations. Randomization-based tests are robust and valid alternatives to likelihood-based tests and should be considered more frequently by clinical investigators.

A randomized controlled trial comparing non-steroidal anti-inflammatory and fusion protein inhibitors singly and in combination on the histopathology of bovine respiratory syncytial virus infection

Bovine respiratory syncytial virus (RSV) has substantial morbidity in young calves, and closely parallels human RSV in infants. We performed a randomized controlled trial in five to six-week-old Holstein calves (Bos taurus). comparing fusion protein inhibitor (FPI) and non-steroidal anti-inflammatory drug (NSAID) singly and in combination at three and five days after experimental BRSV infection. Thirty-six calves received one of six treatments; Ibuprofen started on day 3, Ibuprofen started on day 5, FPI started on day 5, FPI and Ibuprofen started on day 3, FPI and Ibuprofen started on day 5, or placebo. We have previously reported significant clinical benefits when combined FPI and NSAID treatment was started at three and five days after bovine RSV infection. Necropsy was performed on Day 10 following infection and hematoxylin and eosin staining was performed on sections from each lobe. Histology was described using a four-point scale. We performed canonical discrimination analysis (CDA) to determine the structural level where differences between treatments occurred and mixed effects regression to estimate effect sizes. Separation from placebo was maximal for dual therapy at the levels of the alveolus, septum, and bronchus in CDA. We found that the clinical benefits of combined FPI and NSAID treatment of BRSV extend at least partially from histopathological changes in the lung when treatment was started three days after infection. We found decreased lung injury when ibuprofen was started as monotherapy on day 3, but not day 5 following infection. Combined therapy with both an FPI and ibuprofen was always better than ibuprofen alone. We did not prove that the clinical benefits seen starting FPI and ibuprofen five days after infection can be solely explained by histopathological differences as identified on H&E staining.

Making Patient-Specific Treatment Decisions Using Prognostic Variables and Utilities of Clinical Outcomes

We argue that well-informed patient-specific decision-making may be carried out as three consecutive tasks: (1) estimating key parameters of a statistical model, (2) using prognostic information to convert these parameters into clinically interpretable values, and (3) specifying joint utility functions to quantify risk-benefit trade-offs between clinical outcomes. Using the management of metastatic clear cell renal cell carcinoma as our motivating example, we explain the role of prognostic covariates that characterize between-patient heterogeneity in clinical outcomes. We show that explicitly specifying the joint utility of clinical outcomes provides a coherent basis for patient-specific decision-making.

Comparision of Low and High Volume of Resistance Training on Body Fat and Blood Biomarkers in Untrained Older Women: A Randomized Clinical Trial

ABSTRACT

Cunha, PM, Tomeleri, CM, Nascimento, MA, Mayhew, JL, Fungari, E, Cyrino, LT, Barbosa, DS, Venturini, D, and Cyrino, ES. Comparision of low and high volume of resistance training on body fat and blood biomarkers in untrained older women: a randomized clinical trial. J Strength Cond Res 35(1): 1-8, 2021-The purpose of this study was to compare the effects of resistance training (RT) performed with 2 different volumes on body fat and blood biomarkers in untrained older women. Sixty-five physically independent older women (≥60 years) were randomly assigned to one of 3 groups: low-volume (LV) training group, high-volume (HV) training group, and a control group. Both training groups performed RT for 12 weeks, using 8 exercises of 10-15 repetitions maximum for each exercise. The low-volume group performed only a single set per exercise, whereas the HV group performed 3 sets. Anthropometric, body fat (%), trunk fat, triglycerides (TG), total cholesterol, low-density lipoprotein cholesterol (LDL-c), high-density lipoprotein cholesterol, very LDL-c (VLDL-c), glucose (GLU), C-reactive protein (CRP), and composite Z-score were measured. The HV group obtained greater improvements compared with the LV group (p < 0.05) for TG (LV = -10.5% vs. HV = -16.6%), VLDL-c (LV = -6.5% vs. HV = -14.8%), GLU (LV = -4.7% vs. HV = -11.1%), CRP (LV = -13.2% vs. HV = -30.8%), % body fat (LV = -2.4% vs. HV = -6.1%), and composite Z-score (LV = -0.13 ± 0.30 vs. HV = -0.57 ± 0.29). Trunk fat was reduced (p < 0.05) only in the HV group (-6.8%). We conclude that RT performed in higher volume seems to be the most appropriate strategy to reduce body fat (%), trunk fat, improve blood biomarkers, and reduce composite Z-score in older women.

Changes in lipoprotein particle subclasses, standard lipids, and apolipoproteins after supplementation with n-3 or n-6 PUFAs in abdominal obesity: A randomized double-blind crossover study

BACKGROUND & AIMS

Marine-derived omega-3 (n-3) polyunsaturated fatty acids (PUFAs), mainly eicosapentaenoic acid (EPA) and docosahexaenoic acid (DHA), lower circulating levels of triacylglycerols (TAGs), and the plant-derived omega-6 (n-6) PUFA linoleic acid (LA) may reduce cholesterol levels. Clinical studies on effects of these dietary or supplemental PUFAs on other blood fat fractions are few and have shown conflicting results. This study aimed to determine effects of high-dose supplemental n-3 (EPA + DHA) and n-6 (LA) PUFAs from high-quality oils on circulating lipoprotein subfractions and standard lipids (primary outcomes), as well as apolipoproteins, fatty acids, and glycemic control (secondary outcomes), in females and males with abdominal obesity.

METHODS

This was a randomized double-blind crossover study with two 7-wk intervention periods separated by a 9-wk washout phase. Females (n = 16) were supplemented with 3 g/d of EPA + DHA (TAG fish oil) or 15 g/d of LA (safflower oil), while males (n = 23) received a dose of 4 g/d of EPA + DHA or 20 g/d of LA. In fasting blood samples, we investigated lipoprotein particle subclasses by nuclear magnetic resonance spectroscopy, as well as standard lipids, apolipoproteins, fatty acid profiles, and glucose and insulin. Data were analyzed by linear mixed-effects modeling with 'subjects' as the random factor.

RESULTS

The difference between interventions in relative change scores was among the lipoprotein subfractions significant for total very-low-density lipoproteins (VLDLs) (n-3 vs. n-6: -38%∗ vs. +16%, p < 0.001; ∗: significant within-treatment change score), large VLDLs (-58%∗ vs. -0.91%, p < 0.001), small VLDLs (-57%∗ vs. +41%∗, p < 0.001), total low-density lipoproteins (LDLs) (+5.8%∗ vs. -4.3%∗, p = 0.002), large LDLs (+23%∗ vs. -2.1%, p = 0.004), total high-density lipoproteins (HDLs) (-6.0%∗ vs. +3.7%, p < 0.001), large HDLs (+11%∗ vs. -5.3%, p = 0.001), medium HDLs (-24%∗ vs. +6.2%, p = 0.030), and small HDLs (-9.9%∗ vs. +9.6%∗, p = 0.002), and among standard lipids for TAGs (-16%∗ vs. -2.6%, p = 0.014), non-esterified fatty acids (-19%∗ vs. +5.5%, p = 0.033), and total cholesterol (-0.28% vs. -4.4%∗, p = 0.042). A differential response in relative change scores was also found for apolipoprotein (apo)B (+0.40% vs. -6.0%∗, p = 0.008), apoA-II (-6.0%∗ vs. +1.5%, p = 0.001), apoC-II (-11%∗ vs. -1.7%, p = 0.025), and apoE (+3.3% vs. -3.8%, p = 0.028).

CONCLUSIONS

High-dose supplementation of high-quality oils with n-3 (EPA + DHA) or n-6 (LA) PUFAs was followed by reductions in primarily TAG- or cholesterol-related markers, respectively. The responses after both interventions point to changes in the lipoprotein-lipid-apolipoprotein profile that have been associated with reduced cardiometabolic risk, also among people with TAG or LDL-C levels within the normal range.

REGISTRATION

Registered under ClinicalTrials.gov Identifier: NCT02647333.

CLINICAL TRIAL REGISTRATION

Registered at https://clinicaltrials.gov/ct2/show/NCT02647333.