Sequential causal inference: application to randomized trials of adaptive treatment strategies

Innovation in the Delivery of Behavioral Health Services

Several factors motivate the need for innovation to improve the delivery of behavioral health services, including increased rates of mental health and substance use disorders, limited access to services, inconsistent use of evidence-based practices, and persistent racial and ethnic disparities. This narrative review identifies promising innovations that address these challenges, assesses empirical evidence for the effectiveness of these innovations and the extent to which they have been adopted and implemented, and suggests next steps for research. We review five categories of innovations: organizational models, including a range of novel locations for providing services and new ways of organizing services within and across sites; information and communication technologies; workforce; treatment technologies; and policy and regulatory changes. We conclude by discussing the need to strengthen and accelerate the contributions of implementation science to close the gap between the launch of innovative behavioral health services and their widespread use.

The Role of Nurses' Uncertainty in Decision-Making Process of Pain Management in People with Dementia

Pain in people with dementia (PWD) is underassessed and undertreated. Treatment of pain in people with dementia goes awry because of poor assessment, poor treatment, and factors related to nursing decision-making skills. Several theoretical models addressed the role of nurses' critical thinking and decision-making skills in pain treatment, like the cognitive continuum theory (CCT) and the adaptive pain management (APT). Only the Response to Certainty of Pain (RCP) model was the first model to posit relationships between nurses' uncertainty, pain assessment, and patient outcomes. Gilmore-Bykovskyi and Bowers developed the RCP, which incorporates the concept of uncertainty and how it relates to the problem of unrelieved pain in PWD. The RCP model has the potential to provide good understanding of the problem of unrelieved pain in people with dementia. It also could help to develop a research study that brings comfort to an often neglected and vulnerable population.

Design and inference for the intent-to-treat principle using adaptive treatment

Nonadherence to assigned treatment jeopardizes the power and interpretability of intent-to-treat comparisons from clinical trial data and continues to be an issue for effectiveness studies, despite their pragmatic emphasis. We posit that new approaches to design need to complement developments in methods for causal inference to address nonadherence, in both experimental and practice settings. This paper considers the conventional study design for psychiatric research and other medical contexts, in which subjects are randomized to treatments that are fixed throughout the trial and presents an alternative that converts the fixed treatments into an adaptive intervention that reflects best practice. The key element is the introduction of an adaptive decision point midway into the study to address a patient's reluctance to remain on treatment before completing a full-length trial of medication. The clinical uncertainty about the appropriate adaptation prompts a second randomization at the new decision point to evaluate relevant options. Additionally, the standard 'all-or-none' principal stratification (PS) framework is applied to the first stage of the design to address treatment discontinuation that occurs too early for a midtrial adaptation. Drawing upon the adaptive intervention features, we develop assumptions to identify the PS causal estimand and to introduce restrictions on outcome distributions to simplify expectation-maximization calculations. We evaluate the performance of the PS setup, with particular attention to the role played by a binary covariate. The results emphasize the importance of collecting covariate data for use in design and analysis. We consider the generality of our approach beyond the setting of psychiatric research.

Introduction to dynamic treatment strategies and sequential multiple assignment randomization

BACKGROUND

In June 2013, a 1-day workshop on Dynamic Treatment Strategies (DTSs) and Sequential Multiple Assignment Randomized Trials (SMARTs) was held at the University of Pennsylvania in Philadelphia, Pennsylvania. These two linked topics have generated a great deal of interest as researchers have recognized the importance of comparing entire strategies for managing chronic disease. A number of articles emerged from that workshop.

PURPOSE

The purpose of this survey of the DTS/SMART methodology (which is taken from the introductory talk in the workshop) is to provide the reader the collected articles presented in this volume with sufficient background to appreciate the more detailed discussions in the articles.

METHODS

The way that the DTS arises naturally in clinical practice is described, along with its connection to the well-known difficulties of interpreting the analysis by intention-to-treat. The SMART methodology for comparing DTS is described, and the basics of estimation and inference presented.

RESULTS

The DTS/SMART methodology can be a flexible and practical way to optimize ongoing clinical decision making, providing evidence (based on randomization) for comparative effectiveness.

LIMITATIONS

The DTS/SMART methodology is not a solution for unstandardized study protocols.

CONCLUSIONS

The DTS/SMART methodology has growing relevance to comparative effectiveness research and the needs of the learning healthcare system.

Modeling of experts' divergent prior beliefs for a sequential phase III clinical trial

BACKGROUND

There have been few Bayesian analyses of phase III sequential clinical trials that model divergent expert opinions in a single distribution.

PURPOSE

We used modeling of experts' opinions to perform additional Bayesian analyses of a randomized clinical trial (designed as a sequential trial), particularly when a bimodal shape is observed. We provide an illustrative example based on a randomized trial conducted in patients aged between 65 and 75 years with multiple myeloma as the case study.

METHODS

The main endpoint of the trial was overall survival (OS). Prior distribution of the log hazard ratio of death in the experimental versus the control arm ( $$\theta $$ ) was constructed based on elicitation of experts using a mixture of normal distributions estimated by the Expectation-Maximisation (EM) algorithm. At each interim and terminal analysis, the posterior probability of $$\theta $$ and the resulting increases in median OS in the experimental arm compared to the control were computed. The results were compared to results obtained using either skeptical, enthusiastic, or a mixture of those priors. Finally, we discuss our results in light of the frequentist approach originally designed for the trial.

RESULTS

A total of 39 experts reported their opinion on the median OS in the experimental arm compared to the median control survival of 30 months. The resulting pooled distribution of the log hazard ratios exhibited a bimodal profile. When the prior mixture of the normal distribution was fitted to the data sets from the experts, 44% of the experts' opinions were optimistic and 56% were doubtful. At the final analysis, the percentage of doubting experts dropped to 18%. This corresponded to a posterior probability of an improved OS in the experimental arm compared to the control arm of at least 0.98, regardless of the prior. These findings are in agreement with the original conclusion of the trial regarding the beneficial effect of the experimental treatment in this population.

LIMITATIONS

Only 39 experts among the 120 questioned physicians responded to the inquiry. Our approach was hybrid because the prior mixture was estimated using the EM algorithm, and a full Bayesian approach may have been used.

CONCLUSIONS

Bayesian inference allows the quantification of increased survival in terms of probability distributions and provides investigators with an additional tool in the analysis of a randomized phase III clinical trial. Using a mixture of densities appears to be a promising strategy for incorporating the bimodal profile of prior opinion, with actualization of the two components along the trial as an illustration of the evolution of opinions as data are accumulated.

Innovative Clinical Trial Designs: Toward a 21st-Century Health Care System

Whereas the 20th-century health care system sometimes seemed to be inhospitable to and unmoved by experimental research, its inefficiency and unaffordability have led to reforms that foreshadow a new health care system. We point out certain opportunities and transformational needs for innovations in study design offered by the 21st-century health care system, and describe some innovative clinical trial designs and novel design methods to address these needs and challenges.

HETEROGENEITY IN TREATMENT EFFECT AND COMPARATIVE EFFECTIVENESS RESEARCH

The ultimate goal of comparative effectiveness research (CER) is to develop and disseminate evidence-based information about which interventions are most effective for which patients under what circumstances. To achieve this goal it is crucial that researchers in methodology development find appropriate methods for detecting the presence and sources of heterogeneity in treatment effect (HTE). Comparing with the typically reported average treatment effect (ATE) in randomized controlled trials and non-experimental (i.e., observational) studies, identifying and reporting HTE better reflect the nature and purposes of CER. Methodologies of CER include meta-analysis, systematic review, design of experiments that encompasses HTE, and statistical correction of various types of estimation bias, which is the focus of this review.

Bayesian adaptive clinical trials: a dream for statisticians only?

Adaptive or 'flexible' designs have emerged, mostly within frequentist frameworks, as an effective way to speed up the therapeutic evaluation process. Because of their flexibility, Bayesian methods have also been proposed for Phase I through Phase III adaptive trials; however, it has been reported that they are poorly used in practice. We aim to describe the international scientific production of Bayesian clinical trials by investigating the actual development and use of Bayesian 'adaptive' methods in the setting of clinical trials. A bibliometric study was conducted using the PubMed and Science Citation Index-Expanded databases. Most of the references found were biostatistical papers from various teams around the world. Most of the authors were from the US, and a large proportion was from the MD Anderson Cancer Center (University of Texas, Houston, TX). The spread and use of these articles depended heavily on their topic, with 3.1% of the biostatistical articles accumulating at least 25 citations within 5 years of their publication compared with 15% of the reviews and 32% of the clinical articles. We also examined the reasons for the limited use of Bayesian adaptive design methods in clinical trials and the areas of current and future research to address these challenges. Efforts to promote Bayesian approaches among statisticians and clinicians appear necessary.

Efficient design and inference for multistage randomized trials of individualized treatment policies

Clinical demand for individualized "adaptive" treatment policies in diverse fields has spawned development of clinical trial methodology for their experimental evaluation via multistage designs, building upon methods intended for the analysis of naturalistically observed strategies. Because often there is no need to parametrically smooth multistage trial data (in contrast to observational data for adaptive strategies), it is possible to establish direct connections among different methodological approaches. We show by algebraic proof that the maximum likelihood (ML) and optimal semiparametric (SP) estimators of the population mean of the outcome of a treatment policy and its standard error are equal under certain experimental conditions. This result is used to develop a unified and efficient approach to design and inference for multistage trials of policies that adapt treatment according to discrete responses. We derive a sample size formula expressed in terms of a parametric version of the optimal SP population variance. Nonparametric (sample-based) ML estimation performed well in simulation studies, in terms of achieved power, for scenarios most likely to occur in real studies, even though sample sizes were based on the parametric formula. ML outperformed the SP estimator; differences in achieved power predominately reflected differences in their estimates of the population mean (rather than estimated standard errors). Neither methodology could mitigate the potential for overestimated sample sizes when strong nonlinearity was purposely simulated for certain discrete outcomes; however, such departures from linearity may not be an issue for many clinical contexts that make evaluation of competitive treatment policies meaningful.

What have we learned about trial design from NIMH-funded pragmatic trials?

At the 2008 annual meeting of the American College of Neuropsychopharmacology (ACNP), a symposium was devoted to the following question: 'what have we learned about the design of pragmatic clinical trials (PCTs) from the recent costly long-term, large-scale trials of psychiatric treatments?' in order to inform the design of future trials. In all, 10 recommendations were generated placing emphasis on (1) appropriate conduct of pragmatic trials; (2) clinical, rather than, merely statistical significance; (3) sampling from the population clinicians are called upon to treat; (4) clinical outcomes of patients, rather than, on outcome measures; (5) use of stratification, controlling, or adjusting when necessary and not otherwise; (6) appropriate consideration of site differences in multisite studies; (7) encouragement of 'post hoc' exploration to generate (not test) hypotheses; (8) precise articulation of the treatment strategy to be tested and use of the corresponding appropriate design; (9) expanded opportunity for training of researchers and reviewers in RCT principles; and (10) greater emphasis on data sharing.

Sample size calculations for evaluating treatment policies in multi-stage designs

BACKGROUND

Sequential multiple assignment randomized (SMAR) designs are used to evaluate treatment policies, also known as adaptive treatment strategies (ATS). The determination of SMAR sample sizes is challenging because of the sequential and adaptive nature of ATS, and the multi-stage randomized assignment used to evaluate them.

PURPOSE

We derive sample size formulae appropriate for the nested structure of successive SMAR randomizations. This nesting gives rise to ATS that have overlapping data, and hence between-strategy covariance. We focus on the case when covariance is substantial enough to reduce sample size through improved inferential efficiency.

METHODS

Our design calculations draw upon two distinct methodologies for SMAR trials, using the equality of the optimal semi-parametric and Bayesian predictive estimators of standard error. This 'hybrid' approach produces a generalization of the t-test power calculation that is carried out in terms of effect size and regression quantities familiar to the trialist.

RESULTS

Simulation studies support the reasonableness of underlying assumptions as well as the adequacy of the approximation to between-strategy covariance when it is substantial. Investigation of the sensitivity of formulae to misspecification shows that the greatest influence is due to changes in effect size, which is an a priori clinical judgment on the part of the trialist.

LIMITATIONS

We have restricted simulation investigation to SMAR studies of two and three stages, although the methods are fully general in that they apply to 'K-stage' trials.

CONCLUSIONS

Practical guidance is needed to allow the trialist to size a SMAR design using the derived methods. To this end, we define ATS to be 'distinct' when they differ by at least the (minimal) size of effect deemed to be clinically relevant. Simulation results suggest that the number of subjects needed to distinguish distinct strategies will be significantly reduced by adjustment for covariance only when small effects are of interest.

Adaptive treatment strategies in chronic disease

An adaptive treatment strategy (ATS) is a rule for adapting a treatment plan to a patient's history of previous treatments and the response to those treatments. The ongoing management of chronic disease defines an ATS, which may be implicit and hidden or explicit and well-specified. The ATS is characterized by the use of intermediate, early markers of response to dynamically alter treatment decisions, in order to achieve a favorable ultimate outcome. We illustrate the ATS concept and describe how the effect of initial treatment decisions depends on the performance of subsequent decisions at later stages. We show how to compare two or more ATSs, or to determine an optimal ATS, using a sequential multiple assignment randomized (SMAR) trial. Designers of clinical trials might find the ATS concept useful in improving the efficiency and ecological relevance of clinical trials.

Developing and testing adaptive treatment strategies using substance-induced psychosis as an example

Decisions concerning treatment changes pervade the management of chronic psychiatric disorders that resist definitive cure, yet empirical evidence for the comparative clinical effectiveness of treatment strategies remains underdeveloped. In this paper we exploit the example of psychosis following substance use to illustrate some new developments in clinical trials design that can provide the most solid evidence base for defining successful strategies. The intent is to explore the strengths and limitations of the methodological approach through a meaningful clinical example, with an emphasis on concepts and issues. Both methodology and clinical science are overviewed.