Noninferiority and equivalence tests in sequential, multiple assignment, randomized trials (SMARTs)

A framework for testing non-inferiority in a three-arm, sequential, multiple assignment randomized trial

Sequential multiple assignment randomized trial design is becoming increasingly used in the field of precision medicine. This design allows comparisons of sequences of adaptive interventions tailored to the individual patient. Superiority testing is usually the initial goal in order to determine which embedded adaptive intervention yields the best primary outcome on average. When direct superiority is not evident, yet an adaptive intervention poses other benefits, then non-inferiority testing is warranted. Non-inferiority testing in the sequential multiple assignment randomized trial setup is rather new and involves the specification of non-inferiority margin and other important assumptions that are often unverifiable internally. These challenges are not specific to sequential multiple assignment randomized trial and apply to two-arm non-inferiority trials that do not include a standard-of-care (or placebo) arm. To address some of these challenges, three-arm non-inferiority trials that include the standard-of-care arm are proposed. However, methods developed so far for three-arm non-inferiority trials are not sequential multiple assignment randomized trial-specific. This is because apart from embedded adaptive interventions, sequential multiple assignment randomized trial typically does not include a third standard-of-care arm. In this article, we consider a three-arm sequential multiple assignment randomized trial from an National Institutes of Health-funded study of symptom management strategies among people undergoing cancer treatment. Motivated by that example, we propose a novel data analytic method for non-inferiority testing in the framework of three-arm sequential multiple assignment randomized trial for the first time. Sample size and power considerations are discussed through extensive simulation studies to elucidate our method.

Comparison of the low-calorie DASH diet and a low-calorie diet on serum TMAO concentrations and gut microbiota composition of adults with overweight/obesity: a randomized control trial

This study compares two diets, Dietary Approaches to Stop Hypertension (DASH) and a Low-Calorie Diet on Trimethylamine N-oxide (TMAO) levels and gut microbiota. 120 obese adults were randomly allocated to these three groups: a low-calorie DASH diet, a Low-Calorie diet, or a control group for 12 weeks. Outcomes included plasma TMAO, lipopolysaccharides (LPS), and gut microbiota profiles. After the intervention, the low-calorie DASH diet group demonstrated a greater decrease in TMAO levels (-20 ± 8.1 vs. -10.63 ± 4.6 μM) and a significant decrease in LPS concentration (-19.76 ± 4.2 vs. -5.68 ± 2.3) compared to the low-calorie diet group. Furthermore, the low-calorie DASH diet showed a higher decrease in the Firmicutes and Bactericides (F/B) ratio, which influenced TMAO levels, compared to the Low-Calorie diet (p = 0.028). The current study found the low-calorie DASH diet improves TMAO and LPS in comparison to a Low-Calorie diet.

Design of experiments with sequential randomizations on multiple timescales: the hybrid experimental design

Psychological interventions, especially those leveraging mobile and wireless technologies, often include multiple components that are delivered and adapted on multiple timescales (e.g., coaching sessions adapted monthly based on clinical progress, combined with motivational messages from a mobile device adapted daily based on the person's daily emotional state). The hybrid experimental design (HED) is a new experimental approach that enables researchers to answer scientific questions about the construction of psychological interventions in which components are delivered and adapted on different timescales. These designs involve sequential randomizations of study participants to intervention components, each at an appropriate timescale (e.g., monthly randomization to different intensities of coaching sessions and daily randomization to different forms of motivational messages). The goal of the current manuscript is twofold. The first is to highlight the flexibility of the HED by conceptualizing this experimental approach as a special form of a factorial design in which different factors are introduced at multiple timescales. We also discuss how the structure of the HED can vary depending on the scientific question(s) motivating the study. The second goal is to explain how data from various types of HEDs can be analyzed to answer a variety of scientific questions about the development of multicomponent psychological interventions. For illustration, we use a completed HED to inform the development of a technology-based weight loss intervention that integrates components that are delivered and adapted on multiple timescales.

SMART Binary: New Sample Size Planning Resources for SMART Studies with Binary Outcome Measurements

Sequential Multiple-Assignment Randomized Trials (SMARTs) play an increasingly important role in psychological and behavioral health research. This experimental approach enables researchers to answer scientific questions about how to sequence and match interventions to the unique, changing needs of individuals. A variety of sample size planning resources for SMART studies have been developed, enabling researchers to plan SMARTs for addressing different types of scientific questions. However, relatively limited attention has been given to planning SMARTs with binary (dichotomous) outcomes, which often require higher sample sizes relative to continuous outcomes. Existing resources for estimating sample size requirements for SMARTs with binary outcomes do not consider the potential to improve power by including a baseline measurement and/or multiple repeated outcome measurements. The current paper addresses this issue by providing sample size planning simulation procedures and approximate formulas for two-wave repeated measures binary outcomes (i.e., two measurement times for the outcome variable, before and after intervention delivery). The simulation results agree well with the formulas. We also discuss how to use simulations to calculate power for studies with more than two outcome measurement occasions. Results show that having at least one repeated measurement of the outcome can substantially improve power under certain conditions.

A novel approach to assess dynamic treatment regimes embedded in a SMART with an ordinal outcome

Sequential multiple assignment randomized trials (SMARTs) are used to construct data-driven optimal intervention strategies for subjects based on their intervention and covariate histories in different branches of health and behavioral sciences where a sequence of interventions is given to a participant. Sequential intervention strategies are often called dynamic treatment regimes (DTR). In the existing literature, the majority of the analysis methodologies for SMART data assume a continuous primary outcome. However, ordinal outcomes are also quite common in clinical practice. In this work, first, we introduce the notion of generalized odds ratio ( G O R $$ GOR $$ ) to compare two DTRs embedded in a SMART with an ordinal outcome and discuss some combinatorial properties of this measure. Next, we propose a likelihood-based approach to estimate G O R $$ GOR $$ from SMART data, and derive the asymptotic properties of its estimate. We discuss alternative ways to estimate G O R $$ GOR $$ using concordant-discordant pairs and two-sample U $$ U $$ -statistic. We derive the required sample size formula for designing SMARTs with ordinal outcomes based on G O R $$ GOR $$ . A simulation study shows the performance of the estimated G O R $$ GOR $$ in terms of the estimated power corresponding to the derived sample size. The methodology is applied to analyze data from the SMART+ study, conducted in the UK, to improve carbohydrate periodization behavior in athletes using a menu planner mobile application, Hexis Performance. A freely available Shiny web app using R is provided to make the proposed methodology accessible to other researchers and practitioners.

Acupuncture as an Adjunct Treatment to Cognitive-Behavioral Therapy for Insomnia

Cognitive-behavioral therapy for insomnia (CBT-I) is the main recommended treatment for patients presenting with insomnia; however, the treatment is not equally effective for all, and several factors can contribute to a diminished treatment response. The rationale for combining CBT-I treatment with acupuncture is explored, and evidence supporting its use in treating insomnia and related comorbidities is discussed. Practical, regulatory, and logistical issues with implementing a combined treatment are examined, and future directions for research are made. Growing evidence supports the effectiveness of acupuncture in treating insomnia and comorbid conditions, and warrants further investigation of acupuncture as an adjunct to CBT-I.

The Sequential Multiple Assignment Randomized Trial for Controlling Infectious Diseases: A Review of Recent Developments

Infectious diseases have posed severe threats to public health across the world. Effective prevention and control of infectious diseases in the long term requires adapting interventions based on epidemiological evidence. The sequential multiple assignment randomized trial (SMART) is a multistage randomized trial that can provide valid evidence of when and how to adapt interventions for controlling infectious diseases based on evolving epidemiological evidence. We review recent developments in SMARTs to bring wider attention to the potential benefits of employing SMARTs in constructing effective adaptive interventions for controlling infectious diseases and other threats to public health. We discuss 2 example SMARTs for infectious diseases and summarize recent developments in SMARTs from the varied aspects of design, analysis, cost, and ethics. Public health investigators are encouraged to familiarize themselves with the related materials we discuss and collaborate with experts in SMARTs to translate the methodological developments into preeminent public health research. (Am J Public Health. 2023;113(1):49-59. https://doi.org/10.2105/AJPH.2022.307135).

Hybrid Experimental Designs for Intervention Development: What, Why, and How

Advances in mobile and wireless technologies offer tremendous opportunities for extending the reach and impact of psychological interventions and for adapting interventions to the unique and changing needs of individuals. However, insufficient engagement remains a critical barrier to the effectiveness of digital interventions. Human delivery of interventions (e.g., by clinical staff) can be more engaging but potentially more expensive and burdensome. Hence, the integration of digital and human-delivered components is critical to building effective and scalable psychological interventions. Existing experimental designs can be used to answer questions either about human-delivered components that are typically sequenced and adapted at relatively slow timescales (e.g., monthly) or about digital components that are typically sequenced and adapted at much faster timescales (e.g., daily). However, these methodologies do not accommodate sequencing and adaptation of components at multiple timescales and hence cannot be used to empirically inform the joint sequencing and adaptation of human-delivered and digital components. Here, we introduce the hybrid experimental design (HED)-a new experimental approach that can be used to answer scientific questions about building psychological interventions in which human-delivered and digital components are integrated and adapted at multiple timescales. We describe the key characteristics of HEDs (i.e., what they are), explain their scientific rationale (i.e., why they are needed), and provide guidelines for their design and corresponding data analysis (i.e., how can data arising from HEDs be used to inform effective and scalable psychological interventions).

Sequential Multiple Assignment Randomized Trial (SMART) to identify optimal sequences of telemedicine interventions for improving initiation of insulin therapy: A simulation study

BACKGROUND

To examine the value of a Sequential Multiple Assignment Randomized Trial (SMART) design compared to a conventional randomized control trial (RCT) for telemedicine strategies to support titration of insulin therapy for Type 2 Diabetes Mellitus (T2DM) patients new to insulin.

METHODS

Microsimulation models were created in R using a synthetic sample based on primary data from 63 subjects enrolled in a pilot study of a smartphone application (App), Diabetes Pal compared to a nurse-based telemedicine strategy (Nurse). For comparability, the SMART and an RCT design were constructed to allow comparison of four (embedded) adaptive interventions (AIs).

RESULTS

In the base case scenario, the SMART has similar overall mean expected HbA1c and cost per subject compared with RCT, for sample size of n = 100 over 10,000 simulations. SMART has lower (better) standard deviations of the mean expected HbA1c per AI, and higher efficiency of choosing the correct AI across various sample sizes. The differences between SMART and RCT become apparent as sample size decreases. For both trial designs, the threshold value at which a subject was deemed to have been responsive at an intermediate point in the trial had an optimal choice (i.e., the sensitivity curve had a U-shape). SMART design dominates the RCT, in the overall mean HbA1c (lower value) when the threshold value is close to optimal.

CONCLUSIONS

SMART is suited to evaluating the efficacy of different sequences of treatment options, in addition to the advantage of providing information on optimal treatment sequences.