Optimal design for inference on the threshold of a biomarker

Enrichment designs with a continuous biomarker require the estimation of a threshold to determine the subpopulation benefitting from the treatment. This article provides the optimal allocation for inference in a two-stage enrichment design for treatment comparisons when a continuous biomarker is suspected to affect patient response. Several design criteria, associated with different trial objectives, are optimized under balanced or Neyman allocation and under equality of the first two empirical biomarker's moments. Moreover, we propose a new covariate-adaptive randomization procedure that converges to the optimum with the fastest available rate. Theoretical and simulation results show that this strategy improves the efficiency of a two-stage enrichment clinical trial, especially with smaller sample sizes and under heterogeneous responses.

Sample size considerations for assessing treatment effect heterogeneity in randomized trials with heterogeneous intracluster correlations and variances

An important consideration in the design and analysis of randomized trials is the need to account for outcome observations being positively correlated within groups or clusters. Two notable types of designs with this consideration are individually randomized group treatment trials and cluster randomized trials. While sample size methods for testing the average treatment effect are available for both types of designs, methods for detecting treatment effect modification are relatively limited. In this article, we present new sample size formulas for testing treatment effect modification based on either a univariate or multivariate effect modifier in both individually randomized group treatment and cluster randomized trials with a continuous outcome but any types of effect modifier, while accounting for differences across study arms in the outcome variance, outcome intracluster correlation coefficient (ICC) and the cluster size. We consider cases where the effect modifier can be measured at either the individual level or cluster level, and with a univariate effect modifier, our closed-form sample size expressions provide insights into the optimal allocation of groups or clusters to maximize design efficiency. Overall, our results show that the required sample size for testing treatment effect heterogeneity with an individual-level effect modifier can be affected by unequal ICCs and variances between arms, and accounting for such between-arm heterogeneity can lead to more accurate sample size determination. We use simulations to validate our sample size formulas and illustrate their application in the context of two real trials: an individually randomized group treatment trial (the AWARE study) and a cluster randomized trial (the K-DPP study).

Best (but oft forgotten) practices: Efficient sample sizes for commonly used trial designs

Designing studies such that they have a high level of power to detect an effect or association of interest is an important tool to improve the quality and reproducibility of findings from such studies. Since resources (research subjects, time, and money) are scarce, it is important to obtain sufficient power with minimum use of such resources. For commonly used randomized trials of the treatment effect on a continuous outcome, designs are presented that minimize the number of subjects or the amount of research budget when aiming for a desired power level. This concerns the optimal allocation of subjects to treatments and, in case of nested designs such as cluster-randomized trials and multicenter trials, also the optimal number of centers versus the number of persons per center. Since such optimal designs require knowledge of parameters of the analysis model that are not known in the design stage, in particular outcome variances, maximin designs are presented. These designs guarantee a prespecified power level for plausible ranges of the unknown parameters and minimize research costs for the worst-case values of these parameters. The focus is on a 2-group parallel design, the AB/BA crossover design, and cluster-randomized and multicenter trials with a continuous outcome. How to calculate sample sizes for maximin designs is illustrated for examples from nutrition. Several computer programs that are helpful in calculating sample sizes for optimal and maximin designs are discussed as well as some results on optimal designs for other types of outcomes.

A comparison of the multilevel MIMIC model to the multilevel regression and mixed ANOVA model for the estimation and testing of a cross-level interaction effect: A simulation study

When observing data on a patient-reported outcome measure in, for example, clinical trials, the variables observed are often correlated and intended to measure a latent variable. In addition, such data are also often characterized by a hierarchical structure, meaning that the outcome is repeatedly measured within patients. To analyze such data, it is important to use an appropriate statistical model, such as structural equation modeling (SEM). However, researchers may rely on simpler statistical models that are applied to an aggregated data structure. For example, correlated variables are combined into one sum score that approximates a latent variable. This may have implications when, for example, the sum score consists of indicators that relate differently to the latent variable being measured. This study compares three models that can be applied to analyze such data: the multilevel multiple indicators multiple causes (ML-MIMIC) model, a univariate multilevel model, and a mixed analysis of variance (ANOVA) model. The focus is on the estimation of a cross-level interaction effect that presents the difference over time on the patient-reported outcome between two treatment groups. The ML-MIMIC model is an SEM-type model that considers the relationship between the indicators and the latent variable in a multilevel setting, whereas the univariate multilevel and mixed ANOVA model rely on sum scores to approximate the latent variable. In addition, the mixed ANOVA model uses aggregated second-level means as outcome. This study showed that the ML-MIMIC model produced unbiased cross-level interaction effect estimates when the relationships between the indicators and the latent variable being measured varied across indicators. In contrast, under similar conditions, the univariate multilevel and mixed ANOVA model underestimated the cross-level interaction effect.

Maximin design of cluster randomized trials with heterogeneous costs and variances

Cluster randomized trials evaluate the effect of a treatment on persons nested within clusters, with clusters being randomly assigned to treatment. The optimal sample size at the cluster and person level depends on the study cost per cluster and per person, and the outcome variance at the cluster and the person level. The variances are unknown in the design stage and can differ between treatment arms. As a solution, this paper presents a Maximin design that maximizes the minimum relative efficiency (relative to the optimal design) over the variance parameter space, for trials with two treatment arms and a quantitative outcome. This maximin relative efficiency design (MMRED) is compared with a published Maximin design which maximizes the minimum efficiency (MMED). Both designs are also compared with the optimal designs for homogeneous costs and variances (balanced design) and heterogeneous costs and homogeneous variances (cost-conscious design), for a range of variances based upon three published trials. Whereas the MMED is balanced under high uncertainty about the treatment-to-control variance ratio, the MMRED then tends towards a balanced budget allocation between arms, leading to an unbalanced sample size allocation if costs are heterogeneous, similar to the cost-conscious design. Further, the MMRED corresponds to an optimal design for an intraclass correlation (ICC) in the lower half of the assumed ICC range (optimistic), whereas the MMED is the optimal design for the maximum ICC within the ICC range (pessimistic). Attention is given to the effect of the Welch-Satterthwaite degrees of freedom for treatment effect testing on the design efficiencies.

Optimal allocation to treatments in a sequential multiple assignment randomized trial

One of the main questions in the design of a trial is how many subjects should be assigned to each treatment condition. Previous research has shown that equal randomization is not necessarily the best choice. We study the optimal allocation for a novel trial design, the sequential multiple assignment randomized trial, where subjects receive a sequence of treatments across various stages. A subject's randomization probabilities to treatments in the next stage depend on whether he or she responded to treatment in the current stage. We consider a prototypical sequential multiple assignment randomized trial design with two stages. Within such a design, many pairwise comparisons of treatment sequences can be made, and a multiple-objective optimal design strategy is proposed to consider all such comparisons simultaneously. The optimal design is sought under either a fixed total sample size or a fixed budget. A Shiny App is made available to find the optimal allocations and to evaluate the efficiency of competing designs. As the optimal design depends on the response rates to first-stage treatments, maximin optimal design methodology is used to find robust optimal designs. The proposed methodology is illustrated using a sequential multiple assignment randomized trial example on weight loss management.

Optimal design of cluster randomized trials allowing unequal allocation of clusters and unequal cluster size between arms

There are sometimes cost, scientific, or logistical reasons to allocate individuals unequally in an individually randomized trial. In cluster randomized trials we can allocate clusters unequally and/or allow different cluster size between trial arms. We consider parallel group designs with a continuous outcome, and optimal designs that require the smallest number of individuals to be measured given the number of clusters. Previous authors have derived the optimal allocation ratio for clusters under different variance and/or intracluster correlations (ICCs) between arms, allowing different but prespecified cluster sizes by arm. We derive closed-form expressions to identify the optimal proportions of clusters and of individuals measured for each arm, thereby defining optimal cluster sizes, when cluster size can be chosen freely. When ICCs differ between arms but the variance is equal, the optimal design allocates more than half the clusters to the arm with the higher ICC, but (typically only slightly) less than half the individuals and hence a smaller cluster size. We also describe optimal design under constraints on the number of clusters or cluster size in one or both arms. This methodology allows trialists to consider a range for the number of clusters in the trial and for each to identify the optimal design. Except if there is clear prior evidence for the ICC and variance by arm, a range of values will need to be considered. Researchers should choose a design with adequate power across the range, while also keeping enough clusters in each arm to permit the intended analysis method.

Optimal allocations for two treatment comparisons within the proportional odds cumulative logits model

This paper studies optimal treatment allocations for two treatment comparisons when the outcome is ordinal and analyzed by a proportional odds cumulative logits model. The variance of the treatment effect estimator is used as optimality criterion. The optimal design is sought so that this variance is minimal for a given total sample size or a given budget, meaning that the power for the test on treatment effect is maximal, or it is sought so that a required power level is achieved at a minimal total sample size or budget. Results are presented for three, five and seven ordered response categories, three treatment effect sizes and a skewed, bell-shaped or polarized distribution of the response probabilities. The optimal proportion subjects in the intervention condition decreases with the number of response categories and the costs for the intervention relative to those for the control. The relation between the optimal proportion and effect size depends on the distribution of the response probabilities. The widely used balanced design is not always the most efficient; its efficiency as compared to the optimal design decreases with increasing cost ratio. The optimal design is highly robust to misspecification of the response probabilities and treatment effect size. The optimal design methodology is illustrated using two pharmaceutical examples. A Shiny app is available to find the optimal treatment allocation, to evaluate the efficiency of the balanced design and to study the relation between budget or sample size and power.

Optimal design of cluster randomised trials with continuous recruitment and prospective baseline period

BACKGROUND

Cluster randomised trials, like individually randomised trials, may benefit from a baseline period of data collection. We consider trials in which clusters prospectively recruit or identify participants as a continuous process over a given calendar period, and ask whether and for how long investigators should collect baseline data as part of the trial, in order to maximise precision.

METHODS

We show how to calculate and plot the variance of the treatment effect estimator for different lengths of baseline period in a range of scenarios, and offer general advice.

RESULTS

In some circumstances it is optimal not to include a baseline, while in others there is an optimal duration for the baseline. All other things being equal, the circumstances where it is preferable not to include a baseline period are those with a smaller recruitment rate, smaller intracluster correlation, greater decay in the intracluster correlation over time, or wider transition period between recruitment under control and intervention conditions.

CONCLUSION

The variance of the treatment effect estimator can be calculated numerically, and plotted against the duration of baseline to inform design. It would be of interest to extend these investigations to cluster randomised trial designs with more than two randomised sequences of control and intervention condition, including stepped wedge designs.

Optimal two-stage sampling for mean estimation in multilevel populations when cluster size is informative

To estimate the mean of a quantitative variable in a hierarchical population, it is logistically convenient to sample in two stages (two-stage sampling), i.e. selecting first clusters, and then individuals from the sampled clusters. Allowing cluster size to vary in the population and to be related to the mean of the outcome variable of interest (informative cluster size), the following competing sampling designs are considered: sampling clusters with probability proportional to cluster size, and then the same number of individuals per cluster; drawing clusters with equal probability, and then the same percentage of individuals per cluster; and selecting clusters with equal probability, and then the same number of individuals per cluster. For each design, optimal sample sizes are derived under a budget constraint. The three optimal two-stage sampling designs are compared, in terms of efficiency, with each other and with simple random sampling of individuals. Sampling clusters with probability proportional to size is recommended. To overcome the dependency of the optimal design on unknown nuisance parameters, maximin designs are derived. The results are illustrated, assuming probability proportional to size sampling of clusters, with the planning of a hypothetical survey to compare adolescent alcohol consumption between France and Italy.

A tutorial on sample size calculation for multiple-period cluster randomized parallel, cross-over and stepped-wedge trials using the Shiny CRT Calculator

It has long been recognized that sample size calculations for cluster randomized trials require consideration of the correlation between multiple observations within the same cluster. When measurements are taken at anything other than a single point in time, these correlations depend not only on the cluster but also on the time separation between measurements and additionally, on whether different participants (cross-sectional designs) or the same participants (cohort designs) are repeatedly measured. This is particularly relevant in trials with multiple periods of measurement, such as the cluster cross-over and stepped-wedge designs, but also to some degree in parallel designs. Several papers describing sample size methodology for these designs have been published, but this methodology might not be accessible to all researchers. In this article we provide a tutorial on sample size calculation for cluster randomized designs with particular emphasis on designs with multiple periods of measurement and provide a web-based tool, the Shiny CRT Calculator, to allow researchers to easily conduct these sample size calculations. We consider both cross-sectional and cohort designs and allow for a variety of assumed within-cluster correlation structures. We consider cluster heterogeneity in treatment effects (for designs where treatment is crossed with cluster), as well as individually randomized group-treatment trials with differential clustering between arms, for example designs where clustering arises from interventions being delivered in groups. The calculator will compute power or precision, as a function of cluster size or number of clusters, for a wide variety of designs and correlation structures. We illustrate the methodology and the flexibility of the Shiny CRT Calculator using a range of examples.

Optimal designs for group randomized trials and group administered treatments with outcomes at the subject and group level

With group randomized trials complete groups of subject are randomized to treatment conditions. Such grouping also occurs in individually randomized trials where treatment is administered in groups. Outcomes may be measured at the level of the subject, but also at the level of the group. The optimal design determines the number of groups and the number of subjects per group in the intervention and control conditions. It is found by taking a budgetary constraint into account, where costs are associated with implementing the intervention and control, and with taking measurements on subject and groups. The optimal design is found such that the effect of treatment is estimated with highest efficiency, and the total costs do not exceed the budget that is available. The design that is optimal for the outcome at the subject level is not necessarily optimal for the outcome at the group level. Multiple-objective optimal designs consider both outcomes simultaneously. Their aim is to find a design that has high efficiencies for both outcome measures. An Internet application for finding the multiple-objective optimal design is demonstrated on the basis of an example from smoking prevention in primary education, and another example on consultation time in primary care.

Maximin Efficiencies under Treatment-Dependent Costs and Outcome Variances for Parallel, AA/BB, and AB/BA Designs

If there are no carryover effects, AB/BA crossover designs are more efficient than parallel (A/B) and extended parallel (AA/BB) group designs. This study extends these results in that (a) optimal instead of equal treatment allocation is examined, (b) allowance for treatment-dependent outcome variances is made, and (c) next to treatment effects, also treatment by period interaction effects are examined. Starting from a linear mixed model analysis, the optimal allocation requires knowledge on intraclass correlations in A and B, which typically is rather vague. To solve this, maximin versions of the designs are derived, which guarantee a power level across plausible ranges of the intraclass correlations at the lowest research costs. For the treatment effect, an extensive numerical evaluation shows that if the treatment costs of A and B are equal, or if the sum of the costs of one treatment and measurement per person is less than the remaining subject-specific costs (e.g., recruitment costs), the maximin crossover design is most efficient for ranges of intraclass correlations starting at 0.15 or higher. For other cost scenarios, the maximin parallel or extended parallel design can also become most efficient. For the treatment by period interaction, the maximin AA/BB design can be proven to be the most efficient. A simulation study supports these asymptotic results for small samples.

Efficient design of cluster randomized trials with treatment-dependent costs and treatment-dependent unknown variances

Cluster randomized trials evaluate the effect of a treatment on persons nested within clusters, where treatment is randomly assigned to clusters. Current equations for the optimal sample size at the cluster and person level assume that the outcome variances and/or the study costs are known and homogeneous between treatment arms. This paper presents efficient yet robust designs for cluster randomized trials with treatment‐dependent costs and treatment‐dependent unknown variances, and compares these with 2 practical designs. First, the maximin design (MMD) is derived, which maximizes the minimum efficiency (minimizes the maximum sampling variance) of the treatment effect estimator over a range of treatment‐to‐control variance ratios. The MMD is then compared with the optimal design for homogeneous variances and costs (balanced design), and with that for homogeneous variances and treatment‐dependent costs (cost‐considered design). The results show that the balanced design is the MMD if the treatment‐to control cost ratio is the same at both design levels (cluster, person) and within the range for the treatment‐to‐control variance ratio. It still is highly efficient and better than the cost‐considered design if the cost ratio is within the range for the squared variance ratio. Outside that range, the cost‐considered design is better and highly efficient, but it is not the MMD. An example shows sample size calculation for the MMD, and the computer code (SPSS and R) is provided as supplementary material. The MMD is recommended for trial planning if the study costs are treatment‐dependent and homogeneity of variances cannot be assumed.

Sample size calculations for group randomized trials with unequal group sizes through Monte Carlo simulations

Group randomized trial design is common in cancer prevention and health promotion research with intervention development. Several methods have been developed to handle the design and analytical issues for group randomized trial including the intraclass correlation coefficient. The widely used methods for the sample size calculation for the group randomized trial assume equal sizes across groups. In practice this assumption often fails and group randomized trial studies suffer from considerably lower statistical power than as planned. A few studies have developed sample size calculation methods for unequal group sizes, but most of them are limited to continuous outcomes. In this study, we develop a method for sample size calculation for group randomized trial studies with unequal group sizes based on Monte Carlo simulation in the mixed effect model framework. This approach incorporates the variation of group sizes and can be applied to group randomized trials with different types of outcomes. Further, it is easy to implement and can be applied to most commonly used group randomized trial designs such as pre-and-post cross-sectional and cohort study designs. We demonstrate the application of the proposed approach to two-arm group randomized trial studies with continuous and binary outcomes through simulations and analysis of a real group randomized trial dataset.

Efficient treatment allocation in 2 × 2 cluster randomized trials, when costs and variances are heterogeneous

Typically, clusters and individuals in cluster randomized trials are allocated across treatment conditions in a balanced fashion. This is optimal under homogeneous costs and outcome variances. However, both the costs and the variances may be heterogeneous. Then, an unbalanced allocation is more efficient but impractical as the outcome variance is unknown in the design stage of a study. A practical alternative to the balanced design could be a design optimal for known and possibly heterogeneous costs and homogeneous variances. However, when costs and variances are heterogeneous, both designs suffer from loss of efficiency, compared with the optimal design. Focusing on cluster randomized trials with a 2 × 2 design, the relative efficiency of the balanced design and of the design optimal for heterogeneous costs and homogeneous variances is evaluated, relative to the optimal design. We consider two heterogeneous scenarios (two treatment arms with small, and two with large, costs or variances, or one small, two intermediate, and one large costs or variances) at each design level (cluster, individual, and both). Within these scenarios, we compute the relative efficiency of the two designs as a function of the extents of heterogeneity of the costs and variances, and the congruence (the cheapest treatment has the smallest variance) and incongruence (the cheapest treatment has the largest variance) between costs and variances. We find that the design optimal for heterogeneous costs and homogeneous variances is generally more efficient than the balanced design and we illustrate this theory on a trial that examines methods to reduce radiological referrals from general practices. Copyright © 2016 John Wiley & Sons, Ltd.

Repairing the efficiency loss due to varying cluster sizes in two-level two-armed randomized trials with heterogeneous clustering

In two-armed trials with clustered observations the arms may differ in terms of (i) the intraclass correlation, (ii) the outcome variance, (iii) the average cluster size, and (iv) the number of clusters. For a linear mixed model analysis of the treatment effect, this paper examines the expected efficiency loss due to varying cluster sizes based upon the asymptotic relative efficiency of varying versus constant cluster sizes. Simple, but nearly cost-optimal, correction factors are derived for the numbers of clusters to repair this efficiency loss. In an extensive Monte Carlo simulation, the accuracy of the asymptotic relative efficiency and its Taylor approximation are examined for small sample sizes. Practical guidelines are derived to correct the numbers of clusters calculated under constant cluster sizes (within each treatment) when planning a study. Because of the variety of simulation conditions, these guidelines can be considered conservative but safe in many realistic situations. Copyright © 2016 John Wiley & Sons, Ltd.