Sample Size and Power When Designing a Randomized Trial for the Estimation of Treatment, Selection, and Preference Effects

Likelihood based inferences for trials incorporating participant's treatment choice

Randomized clinical trials are the gold standard for clinical trials as they reduce bias and minimize variability between different arms of a study. One of the drawbacks of these designs is their lack of flexibility to incorporate participant's treatment choice, which may reduce recruitment rates and/or reduce participant's tolerance if they receive a non-preferred treatment. Designs incorporating choice allow a subset of participants to choose their preferred treatment. Most of the current methods to analyze these types of designs are based on an ANOVA approach that do not allow for inclusion of covariates in the model. In this paper, we propose an alternative approach based on likelihood methods that can be used with a broad class of distributions and allow for inclusion of covariates and multiple study arms in the model. Using simulations, we evaluate these methods for a variety of continuous and categorical outcomes. Finally, we illustrate these methods by analyzing change in six minute walking distance from a behavioral intervention study for women with heart disease.

Group sequential two-stage preference designs

The two-stage preference design (TSPD) enables inference for treatment efficacy while allowing for incorporation of patient preference to treatment. It can provide unbiased estimates for selection and preference effects, where a selection effect occurs when patients who prefer one treatment respond differently than those who prefer another, and a preference effect is the difference in response caused by an interaction between the patient's preference and the actual treatment they receive. One potential barrier to adopting TSPD in practice, however, is the relatively large sample size required to estimate selection and preference effects with sufficient power. To address this concern, we propose a group sequential two-stage preference design (GS-TSPD), which combines TSPD with sequential monitoring for early stopping. In the GS-TSPD, pre-planned sequential monitoring allows investigators to conduct repeated hypothesis tests on accumulated data prior to full enrollment to assess study eligibility for early trial termination without inflating type I error rates. Thus, the procedure allows investigators to terminate the study when there is sufficient evidence of treatment, selection, or preference effects during an interim analysis, thereby reducing the design resource in expectation. To formalize such a procedure, we verify the independent increments assumption for testing the selection and preference effects and apply group sequential stopping boundaries from the approximate sequential density functions. Simulations are then conducted to investigate the operating characteristics of our proposed GS-TSPD compared to the traditional TSPD. We demonstrate the applicability of the design using a study of Hepatitis C treatment modality.

Factors associated with choice of behavioural weight loss program by adults with obesity

We assessed the preference for two behavioural weight loss programs, Diabetes Prevention Program (DPP) and Healthy Weight for Living (HWL) in adults with obesity. A cross-sectional survey was fielded on the Amazon Mechanical Turk. Eligibility criteria included reporting BMI ≥30 and at least two chronic health conditions. Participants read about the programs, selected their preferred program, and answered follow-up questions. The estimated probability of choosing either program was not significantly different from .5 (N = 1005, 50.8% DPP and 49.2% HWL, p = .61). Participants' expectations about adherence, weight loss magnitude, and dropout likelihood were associated with their choice (p < .0001). Non-White participants (p = .040) and those with monthly income greater than $4999 (p = .002) were less likely to choose DPP. Participants who had postgraduate education (p = .007), did not report high serum cholesterol (p = .028), and reported not having tried losing weight before (p = .025) were more likely to choose DPP. Those who chose HWL were marginally more likely to report that being offered two different programs rather than one would likely affect their decision to enrol in one of the two (p = .052). The enrolment into DPP and HWL was balanced, but race, educational attainment, income, previous attempt to lose weight, and serum cholesterol levels had significant associations with the choice of weight loss program.

Analysis of survival outcomes using likelihood ratio test in trials incorporating patient's treatment choice

Methods for designing and analyzing multiple arms survival trials that incorporate patient's treatment choice are needed. In these trials, patients are randomized into two groups, random and choice. Participants in the choice group choose their treatment, which is not a current standard practice in randomized clinical trials. In this paper, we propose a new method based on the likelihood function to design and analyze these trials with time to event outcomes in the presence of non-informative right censoring. We use simulations to evaluate the methods for Weibull outcomes, complete and censored. Finally, we provide an illustration for designing a study in which we discuss some design considerations and demonstrate the methods.

Taking a chance: How likely am I to receive my preferred treatment in a clinical trial?

Researchers should ideally conduct clinical trials under a presumption of clinical equipoise, but in fact trial patients will often prefer one or other of the treatments being compared. Receiving an unblinded preferred treatment may affect the study outcome, possibly beneficially, but receiving a non-preferred treatment may induce 'reluctant acquiescence', and poorer outcomes. Even in blinded trials, patients' primary motivation to enrol may be the chance of potentially receiving a desirable experimental treatment, which is otherwise unavailable. Study designs with a higher probability of receiving a preferred treatment (denoted as 'concordance') will be attractive to potential participants, and investigators, because they may improve recruitment and hence enhance study efficiency. Therefore, it is useful to consider the concordance rates associated with various study designs. We consider this question with a focus on comparing the standard, randomised, two-arm, parallel group design with the two-stage randomised patient preference design and Zelen designs; we also mention the fully randomised and partially randomised patient preference designs. For each of these designs, we evaluate the concordance rate as a function of the proportions randomised to the alternative treatments, the distribution of preferences over treatments, and (for the Zelen designs) the proportion of patients who consent to receive their assigned treatment. We also examine the equity of each design, which we define as the similarity between the concordance rates for participants with different treatment preferences. Finally, we contrast each of the alternative designs with the standard design in terms of gain in concordance and change in equity.

Long-Term Effects of Cognitive-Behavioral Therapy and Yoga for Worried Older Adults

OBJECTIVES

Cognitive-behavioral therapy (CBT) and yoga decrease worry and anxiety. There are no long-term data comparing CBT and yoga for worry, anxiety, and sleep in older adults. The impact of preference and selection on these outcomes is unknown. In this secondary data analysis, we compared long-term effects of CBT by telephone and yoga on worry, anxiety, sleep, depressive symptoms, fatigue, physical function, social participation, and pain; and examined preference and selection effects.

DESIGN

In this randomized preference trial, participants (N = 500) were randomized to a: 1) randomized controlled trial (RCT) of CBT or yoga (n = 250); or 2) preference trial (selected CBT or yoga; n = 250). Outcomes were measured at baseline and Week 37.

SETTING

Community.

PARTICIPANTS

Community-dwelling older adults (age 60+ years).

INTERVENTIONS

CBT (by telephone) and yoga (in-person group classes).

MEASUREMENTS

Penn State Worry Questionnaire - Abbreviated (worry);^1,2 Insomnia Severity Index (sleep);³ PROMIS Anxiety Short Form v1.0 (anxiety);^4,5 Generalized Anxiety Disorder Screener (generalized anxiety);^6,7 and PROMIS-29 (depression, fatigue, physical function, social participation, pain).^8,9 RESULTS: Six months after intervention completion, CBT and yoga RCT participants reported sustained improvements from baseline in worry, anxiety, sleep, depressive symptoms, fatigue, and social participation (no significant between-group differences). Using data combined from the randomized and preference trials, there were no significant preference or selection effects. Long-term intervention effects were observed at clinically meaningful levels for most of the study outcomes.

CONCLUSIONS

CBT and yoga both demonstrated maintained improvements from baseline on multiple outcomes six months after intervention completion in a large sample of older adults.

TRIAL REGISTRATION

www.

CLINICALTRIALS

gov Identifier NCT02968238.

A Randomized Preference Trial Comparing Cognitive-Behavioral Therapy and Yoga for the Treatment of Late-Life Worry: Examination of Impact on Depression, Generalized Anxiety, Fatigue, Pain, Social Participation, and Physical Function

Background

Depression, generalized anxiety, fatigue, diminished physical function, reduced social participation, and pain are common for many older adults and negatively impact quality of life. The purpose of the overall trial was to compare the effects of cognitive-behavioral therapy (CBT) and yoga on late-life worry, anxiety, and sleep; and examine preference and selection effects on these outcomes.

Objective

The present analyses compared effects of the 2 interventions on additional outcomes (depressive symptoms, generalized anxiety symptoms, fatigue, pain interference/intensity, physical function, social participation); and examined whether there are preference and selection effects for these treatments.

Methods

A randomized preference trial of CBT and yoga was conducted in adults ≥60 years who scored ≥26 on the Penn State Worry Questionnaire-Abbreviated (PSWQ-A), recruited from outpatient medical clinics, mailings, and advertisements. Cognitive-behavioral therapy consisted of 10 weekly telephone sessions. Yoga consisted of 20 bi-weekly group yoga classes. Participants were randomized to(1): a randomized controlled trial (RCT) of CBT or yoga (n = 250); or (2) a preference trial in which they selected their treatment (CBT or yoga; n = 250). Outcomes were measured at baseline and post-intervention.

Results

Within the RCT, there were significant between-group differences for both pain interference and intensity. The pain interference score improved more for the CBT group compared with the yoga group [intervention effect of (mean (95% CI) = 2.5 (.5, 4.6), P = .02]. For the pain intensity score, the intervention effect also favored CBT over yoga [.7 (.2, 1.3), P < .01]. Depressive symptoms, generalized anxiety, and fatigue showed clinically meaningful within-group changes in both groups. There were no changes in or difference between physical function or social participation for either group. No preference or selection effects were found.

Conclusion

Both CBT and yoga may be useful for older adults for improving psychological symptoms and fatigue. Cognitive-behavioral therapy may offer even greater benefit than yoga for decreasing pain.

Comparison of cognitive-behavioral therapy and yoga for the treatment of late-life worry: A randomized preference trial

BACKGROUND

The purpose of this study was to compare the effects of cognitive-behavioral therapy (CBT) and yoga on late-life worry, anxiety, and sleep; and examine preference and selection effects on these outcomes.

METHODS

A randomized preference trial of CBT and yoga was conducted in community-dwelling adults 60 years or older, who scored 26 or above on the Penn State Worry Questionnaire-Abbreviated (PSWQ-A). CBT consisted of 10 weekly telephone sessions. Yoga consisted of 20 biweekly group yoga classes. The primary outcome was worry (PSWQ-A); the secondary outcomes were anxiety (PROMIS-Anxiety) and sleep (Insomnia Severity Index [ISI]). We examined both preference effects (average effect for those who received their preferred intervention [regardless of whether it was CBT or yoga] minus the average for those who did not receive their preferred intervention [regardless of the intervention]) and selection effect (which addresses the question of whether there is a benefit to getting to select one intervention over the other, and measures the effect on outcomes of self-selection to a specific intervention).

RESULTS

Five hundred older adults were randomized to the randomized trial (125 each in CBT and yoga) or the preference trial (120 chose CBT; 130 chose yoga). In the randomized trial, the intervention effect of yoga compared with CBT adjusted for baseline psychotropic medication use, gender, and race was 1.6 (-0.2, 3.3), p = .08 for the PSWQ-A. Similar results were observed with PROMIS-Anxiety (adjusted intervention effect: 0.3 [-1.5, 2.2], p = .71). Participants randomized to CBT experienced a greater reduction in the ISI compared with yoga (adjusted intervention effect: 2.4 [1.2, 3.7], p < .01]). Estimated in the combined data set (N = 500), the preference and selection effects were not significant for the PSWQ-A, PROMIS-Anxiety, and ISI. Of the 52 adverse events, only two were possibly related to the intervention. None of the 26 serious adverse events were related to the study interventions.

CONCLUSIONS

CBT and yoga were both effective at reducing late-life worry and anxiety. However, a greater impact was seen for CBT compared with yoga for improving sleep. Neither preference nor selection effects was found.

Two-stage randomized trial design for testing treatment, preference, and self-selection effects for count outcomes

While the traditional clinical trial design lays emphasis on testing the treatment effect between randomly assigned groups, it ignores the role of patient preference for a particular treatment in the trial. Yet, for healthcare providers who seek to optimize the patient-centered treatment strategy, the evaluation of a patient's psychology toward each treatment could be a key consideration. The two-stage randomized trial design allows researchers to test patient's preference and selection effects, in addition to the treatment effect. The current methodology for the two-stage design is limited to continuous and binary outcomes; this article extends the model to include count outcomes. The test statistics for preference, selection, and treatment effects are derived. Closed-form sample size formulae are presented for each effect. Simulations are presented to demonstrate the properties of the unstratified and stratified designs. Finally, we apply methods to the use of antimicrobials at the end of life to demonstrate the applicability of the methods.

Relative efficiencies of alternative preference-based designs for randomised trials

Recent work has shown that outcomes in clinical trials can be affected by which treatment the trial participants would select if they were allowed to do so, and if they do or do not actually receive that treatment. These influences are known as selection and preference effects, respectively. Unfortunately, they cannot be evaluated in conventional, parallel group trials because patient preferences remain unknown. However, several alternative designs have been proposed, to measure and take account of patient preferences. In this paper, we discuss three preference-based designs (the two-stage, fully randomised, and partially randomised designs). In conventional trials, only the treatment effect is estimable, while the preference-based designs have the potential to estimate some or all of the selection and preference effects. The relative efficiency of these designs is affected by several factors, including the proportion of participants who are undecided about treatments, or who are unable or unwilling to state a preference; the relative preference rate between the treatments being compared, among patients who do have a preference; and the ratio of patients randomised to each treatment. We also discuss the advantages and disadvantages of these designs under different scenarios.

Accounting for health literacy and intervention preferences when reducing unhealthy snacking: protocol for an online randomised controlled trial

INTRODUCTION

Health literacy describes the cognitive and social skills that individuals use to access, understand and act on health information. Health literacy interventions typically take the 'universal precautions approach' where all consumers are presented with simplified materials. Although this approach can improve knowledge and comprehension, its impact on complex behaviours is less clear. Systematic reviews also suggest that health literacy interventions underuse volitional strategies (such as planning) that play an important role in behaviour change. A recent study found volitional strategies may need to be tailored to the participant's health literacy. The current study aims to replicate these findings in a sample of people who have diabetes and/or are overweight or obese as measured by body mass index, and to investigate the most effective method of allocating an action plan to a participant to reduce unhealthy snacking.

METHODS AND ANALYSIS

We plan to recruit approximately 2400 participants at baseline. Participants will receive one of two alternative online action plans intended to reduce unhealthy snacking ('standard' action plan or 'literacy-sensitive' action plan). Participants will be randomised to a method of allocation to an action plan: (1) random allocation; (2) allocation by health literacy screening tool or (3) allocation by participant selection. Primary outcome is self-reported serves of unhealthy snacks during the previous month. Multiple linear regression will evaluate the impact of health literacy on intervention effectiveness. The analysis will also identify independent contributions of each action plan, method of allocation, health literacy and participant selections on unhealthy snacking at 4-week follow-up.

ETHICS AND DISSEMINATION

This study was approved by the University of Sydney Human Research Ethics Committee (2017/793). Findings will be disseminated through peer-reviewed international journals, conferences and updates with collaborating public health bodies (Diabetes New South Wales (NSW) & Australian Capital Territory (ACT), and Western Sydney Local Health District).

TRIAL REGISTRATION NUMBER

ACTRN12618001409268; Pre-results.

Optimising the two-stage randomised trial design when some participants are indifferent in their treatment preferences

Outcomes in a clinical trial can be affected by any underlying preferences that its participants have for the treatments under comparison and by whether they actually receive their preferred treatment. These effects cannot be evaluated in standard trial designs but are estimable in the alternative two-stage randomised trial design, in which some patients can choose their treatment, while the rest are randomly assigned. We have previously shown that, when all two-stage trial participants have a preferred treatment, the preference effects can be evaluated, in addition to the usual direct effect of treatment. We also determined criteria by which to optimise how many participants should be given a choice of treatment vs being randomised. More recently, we extended our methodology to allow for participants who are unable or unwilling to express a treatment preference if they are assigned to the choice group. In this paper, we show how to optimise the two-stage design when some participants are undecided about their treatment. We demonstrate that the undecided group should be regarded as distinct in the analysis, to obtain valid estimates of the preference effects. We derive the optimal proportion of participants who should be offered a choice of treatment, which in many cases will be close to 50%. More generally, the optima depend on the preference rates for treatments and the proportion of undecided participants, and the parameters of primary interest. We discuss some advantages and disadvantages of the two-stage trial design in this situation and describe a practical example.

Preference option randomized design (PORD) for comparative effectiveness research: Statistical power for testing comparative effect, preference effect, selection effect, intent-to-treat effect, and overall effect

Comparative effectiveness research trials in real-world settings may require participants to choose between preferred intervention options. A randomized clinical trial with parallel experimental and control arms is straightforward and regarded as a gold standard design, but by design it forces and anticipates the participants to comply with a randomly assigned intervention regardless of their preference. Therefore, the randomized clinical trial may impose impractical limitations when planning comparative effectiveness research trials. To accommodate participants' preference if they are expressed, and to maintain randomization, we propose an alternative design that allows participants' preference after randomization, which we call a "preference option randomized design (PORD)". In contrast to other preference designs, which ask whether or not participants consent to the assigned intervention after randomization, the crucial feature of preference option randomized design is its unique informed consent process before randomization. Specifically, the preference option randomized design consent process informs participants that they can opt out and switch to the other intervention only if after randomization they actively express the desire to do so. Participants who do not independently express explicit alternate preference or assent to the randomly assigned intervention are considered to not have an alternate preference. In sum, preference option randomized design intends to maximize retention, minimize possibility of forced assignment for any participants, and to maintain randomization by allowing participants with no or equal preference to represent random assignments. This design scheme enables to define five effects that are interconnected with each other through common design parameters-comparative, preference, selection, intent-to-treat, and overall/as-treated-to collectively guide decision making between interventions. Statistical power functions for testing all these effects are derived, and simulations verified the validity of the power functions under normal and binomial distributions.

An alternative method to analyse the biomarker-strategy design

Recent developments in genomics and proteomics enable the discovery of biomarkers that allow identification of subgroups of patients responding well to a treatment. One currently used clinical trial design incorporating a predictive biomarker is the so‐called biomarker strategy design (or marker‐based strategy design). Conventionally, the results from this design are analysed by comparing the mean of the biomarker‐led arm with the mean of the randomised arm. Several problems regarding the analysis of the data obtained from this design have been identified in the literature. In this paper, we show how these problems can be resolved if the sample sizes in the subgroups fulfil the specified orthogonality condition. We also propose a different analysis strategy that allows definition of test statistics for the biomarker‐by‐treatment interaction effect as well as for the classical treatment effect and the biomarker effect. We derive equations for the sample size calculation for the case of perfect and imperfect biomarker assays. We also show that the often used 1:1 randomisation does not necessarily lead to the smallest sample size. In addition, we provide point estimators and confidence intervals for the treatment effects in the subgroups. Application of our method is illustrated using a real data example.

Extensions to the two-stage randomized trial design for testing treatment, self-selection, and treatment preference effects to binary outcomes

While traditional clinical trials seek to determine treatment efficacy within a specified population, they often ignore the role of a patient's treatment preference on his or her treatment response. The two-stage (doubly) randomized preference trial design provides one approach for researchers seeking to disentangle preference effects from treatment effects. Currently, this two-stage design is limited to the design and analysis of continuous outcome variables; in this presentation, we extend this current design to include binary variables. We present test statistics for testing preference, selection, and treatment effects in a two-stage randomized design with a binary outcome measure, with and without stratification. We also derive closed-form sample size formulas to indicate the number of patients needed to detect each effect. A series of simulation studies explore the properties and efficiency of both the unstratified and stratified two-stage randomized trial designs. Finally, we demonstrate the applicability of these methods using an example of a trial of Hepatitis C treatment.

A randomized preference trial of cognitive-behavioral therapy and yoga for the treatment of worry in anxious older adults

Background

Worry is a common problem among older adults. Cognitive-behavioral therapy is the most studied nonpharmacological intervention and it has demonstrated efficacy in reducing late-life worry and anxiety. Although the evidence-base is smaller, yoga has been shown to reduce anxiety and stress. However, little is known about the relative effectiveness of these two nonpharmacological interventions. Further, the impact of patient preference on outcomes is unknown.

Purpose: The purpose to this study is to compare the effectiveness of cognitive-behavioral therapy (CBT) with yoga for improving late-life worry, anxiety, and sleep. We will also examine the effects of preference and selection on outcomes, adherence, and attrition.

Methods

We are conducting a two-stage randomized preference trial comparing CBT and yoga for the reduction of worry in a sample of anxious older adults. Five hundred participants will be randomized to either the preference trial (participants choose the intervention; N = 250) or to the randomized trial (participants are randomized to one of the two interventions; N = 250) with equal probability. CBT consists of 10 telephone-based sessions with an accompanying workbook. Yoga consists of 10 weeks of group yoga classes (twice a week) that is modified for use with older adults.

Conclusions

The study design is based on feedback from anxious older adults who wanted more nonpharmacological options for intervention as well as more input into the intervention they receive. It is the first head-to-head comparison of CBT and yoga for reducing late-life worry and anxiety. It will also provide information about how intervention preference affects outcomes.

Trial registration

ClinicalTrials.gov NCT02968238.

Estimation of treatment preference effects in clinical trials when some participants are indifferent to treatment choice

Background

In the two-stage randomised trial design, a randomly sampled subset of study participants are permitted to choose their own treatment, while the remaining participants are randomised to treatment in the usual way. Appropriate analysis of the data from both arms of the study allows investigators to estimate the impact on study outcomes of treatment preferences that patients may have, in addition to evaluating the usual direct effect of treatment. In earlier work, we showed how to optimise this design by making a suitable choice of the proportion of participants who should be assigned to the choice arm of the trial. However, we ignored the possibility of some participants being indifferent to the treatments under study. In this paper, we extend our earlier work to consider the analysis of two-stage randomised trials when some participants have no treatment preference, even if they are assigned to the choice arm and allowed to choose.

Methods

We compare alternative characterisations of the response profiles of the indifferent or undecided participants, and derive estimates of the treatment and preference effects on study outcomes. We also present corresponding test statistics for these parameters. The methods are illustrated with data from a clinical trial contrasting medical and surgical interventions.

Results

Expressions are obtained to estimate and test the impact of treatment choices on study outcomes, as well as the impact of the actual treatment received. Contrasts are defined between patients with stated treatment preferences and those with no preference. Alternative assumptions concerning the outcomes of undecided participants are described, and an approach leading to unbiased estimation and testing is identified.

Conclusions

Use of the two-stage design can provide important insights into determinants of study outcomes that are not identifiable with other designs. The design can remain attractive even in the presence of participants with no stated treatment preference.

Electronic supplementary material

The online version of this article (doi:10.1186/s12874-017-0304-x) contains supplementary material, which is available to authorized users.

Sample size and power for a stratified doubly randomized preference design

The two-stage (or doubly) randomized preference trial design is an important tool for researchers seeking to disentangle the role of patient treatment preference on treatment response through estimation of selection and preference effects. Up until now, these designs have been limited by their assumption of equal preference rates and effect sizes across the entire study population. We propose a stratified two-stage randomized trial design that addresses this limitation. We begin by deriving stratified test statistics for the treatment, preference, and selection effects. Next, we develop a sample size formula for the number of patients required to detect each effect. The properties of the model and the efficiency of the design are established using a series of simulation studies. We demonstrate the applicability of the design using a study of Hepatitis C treatment modality, specialty clinic versus mobile medical clinic. In this example, a stratified preference design (stratified by alcohol/drug use) may more closely capture the true distribution of patient preferences and allow for a more efficient design than a design which ignores these differences (unstratified version).