Inference about the expected performance of a data-driven dynamic treatment regime

Relative contrast estimation and inference for treatment recommendation

When there are resource constraints, it may be necessary to rank individualized treatment benefits to facilitate the prioritization of assigning different treatments. Most existing literature on individualized treatment rules targets absolute conditional treatment effect differences as a metric for the benefit. However, there can be settings where relative differences may better represent such benefit. In this paper, we consider modeling such relative differences formed as scale-invariant contrasts between the conditional treatment effects. By showing that all scale-invariant contrasts are monotonic transformations of each other, we posit a single index model for a particular relative contrast. We then characterize semiparametric estimating equations, including the efficient score, to estimate index parameters. To achieve semiparametric efficiency, we propose a two-step approach that minimizes a doubly robust loss function for initial estimation and then performs a one-step efficiency augmentation procedure. Careful theoretical and numerical studies are provided to show the superiority of our proposed approach.

Estimation and evaluation of individualized treatment rules following multiple imputation

An individualized treatment rule (ITR) is a function that inputs patient-level information and outputs a recommended treatment. An important focus of precision medicine is to develop optimal ITRs that maximize a population-level distributional summary. However, guidance for estimating and evaluating optimal ITRs in the presence of missing data is limited. Our work is motivated by the Social Incentives to Encourage Physical Activity and Understand Predictors (STEP UP) study. Participants were randomized to a control or one of three interventions designed to increase physical activity and were given wearable devices to record daily steps as a measure of physical activity. Many participants were missing at least one daily step count during the study period. In the primary analysis of the STEP UP trial, multiple imputation (MI) was used to address missingness in daily step counts. Despite ubiquitous use of MI in practice, it has been given relatively little attention in the context of personalized medicine. We fill this gap by describing two frameworks for estimation and evaluation of an optimal ITR following MI and assessing their performance using simulated data. One framework relies on splitting the data into independent training and testing sets for estimation and evaluation, respectively. The other framework estimates an optimal ITR using the full data and constructs anm $$ m $$ -out-of-n $$ n $$ bootstrap confidence interval to evaluate its performance. Finally, we provide an illustrative analysis to estimate and evaluate an optimal ITR from the STEP UP data with a focus on practical considerations such as choosing the number of imputations.

On estimation and cross-validation of dynamic treatment regimes with competing risks

The optimal moment to start renal replacement therapy in a patient with acute kidney injury (AKI) remains a challenging problem in intensive care nephrology. Multiple randomized controlled trials have tried to answer this question, but these contrast only a limited number of treatment initiation strategies. In view of this, we use routinely collected observational data from the Ghent University Hospital intensive care units (ICUs) to investigate different prespecified timing strategies for renal replacement therapy initiation based on time-updated levels of serum potassium, pH, and fluid balance in critically ill patients with AKI with the aim to minimize 30-day ICU mortality. For this purpose, we apply statistical techniques for evaluating the impact of specific dynamic treatment regimes in the presence of ICU discharge as a competing event. We discuss two approaches, a nonparametric one - using an inverse probability weighted Aalen-Johansen estimator - and a semiparametric one - using dynamic-regime marginal structural models. Furthermore, we suggest an easy to implement cross-validation technique to assess the out-of-sample performance of the optimal dynamic treatment regime. Our work illustrates the potential of data-driven medical decision support based on routinely collected observational data.

Deeply Tailored Adaptive Interventions to Reduce College Student Drinking: a Real-World Application of Q-Learning for SMART Studies

M-bridge was a sequential multiple assignment randomized trial (SMART) that aimed to develop a resource-efficient adaptive preventive intervention (API) to reduce binge drinking in first-year college students. The main results of M-bridge suggested no difference, on average, in binge drinking between students randomized to APIs versus assessment-only control, but certain elements of the API were beneficial for at-risk subgroups. This paper extends the main results of M-bridge through an exploratory analysis using Q-learning, a novel algorithm from the computer science literature. Specifically, we sought to further tailor the two aspects of the M-bridge APIs to an individual and test whether deep tailoring offers a benefit over assessment-only control. Q-learning is a method to estimate decision rules that assign optimal treatment (i.e., to minimize binge drinking) based on student characteristics. For the first aspect of the M-bridge API (when to offer), we identified the optimal tailoring characteristic post hoc from a set of 20 candidate variables. For the second (how to bridge), we used a known effect modifier from the trial. The results of our analysis are two rules that optimize (1) the timing of universal intervention for each student based on their motives for drinking and (2) the bridging strategy to indicated interventions (i.e., among those who continue to drink heavily mid-semester) based on mid-semester binge drinking frequency. We estimate that this newly tailored API, if offered to all first-year students, would reduce binge drinking by 1 occasion per 2.5 months (95% CI: decrease of 1.45 to 0.28 occasions, p < 0.01) on average. Our analyses demonstrate a real-world implementation of Q-learning for a substantive purpose, and, if replicable in future trials, our results have practical implications for college campuses aiming to reduce student binge drinking.

Estimation and inference on high-dimensional individualized treatment rule in observational data using split-and-pooled de-correlated score

With the increasing adoption of electronic health records, there is an increasing interest in developing individualized treatment rules, which recommend treatments according to patients' characteristics, from large observational data. However, there is a lack of valid inference procedures for such rules developed from this type of data in the presence of high-dimensional covariates. In this work, we develop a penalized doubly robust method to estimate the optimal individualized treatment rule from high-dimensional data. We propose a split-and-pooled de-correlated score to construct hypothesis tests and confidence intervals. Our proposal adopts the data splitting to conquer the slow convergence rate of nuisance parameter estimations, such as non-parametric methods for outcome regression or propensity models. We establish the limiting distributions of the split-and-pooled de-correlated score test and the corresponding one-step estimator in high-dimensional setting. Simulation and real data analysis are conducted to demonstrate the superiority of the proposed method.

A SMART approach to optimizing delivery of an mHealth intervention among cancer survivors with posttraumatic stress symptoms

BACKGROUND/AIMS

Many cancer survivors who received intensive treatment such as hematopoietic stem cell transplantation (HCT) experience posttraumatic stress disorder (PTSD) symptoms. PTSD is associated with lower quality of life and other symptoms that require clinical treatment. The iterative treatment decisions that happen in clinical practice are not adequately represented in traditional randomized controlled trials (RCT) of PTSD treatments. The proposed stepped-care SMART design allows for evaluation of initial response to the Cancer Distress Coach mobile app; adaptive stepped-care interventions; and precision treatment strategies that tailor treatment selection to patient characteristics.

METHODS/DESIGN

HCT survivors (N = 400) reporting PTSD symptoms are being recruited at two cancer centers and randomly assigned to: 1) Cancer Distress Coach app or 2) Usual Care. The app includes educational and cognitive behavioral therapy (CBT)-based activities. Four weeks post-randomization, participants re-rate their PTSD symptoms and, based on intervention response, non-responders are re-randomized to receive video-conferenced sessions with a therapist: 3) coaching sessions in using the mobile app; or 4) CBT specific to HCT survivors. Participants complete outcome measures of PTSD, depression, and anxiety after Months 1, 3, and 6. Participant characteristics moderating intervention responses will be examined.

CONCLUSIONS

This novel adaptive trial design will afford evidence that furthers knowledge about optimizing PTSD interventions for HCT survivors. To our knowledge, this study is the first SMART design evaluating PTSD symptom management in cancer survivors. If successful, it could be used to optimize treatment among a range of cancer and other trauma survivors.

Evaluating the Effectiveness of Personalized Medicine With Software

We present methodological advances in understanding the effectiveness of personalized medicine models and supply easy-to-use open-source software. Personalized medicine involves the systematic use of individual patient characteristics to determine which treatment option is most likely to result in a better average outcome for the patient. Why is personalized medicine not done more in practice? One of many reasons is because practitioners do not have any easy way to holistically evaluate whether their personalization procedure does better than the standard of care, termed improvement. Our software, "Personalized Treatment Evaluator" (the R package PTE), provides inference for improvement out-of-sample in many clinical scenarios. We also extend current methodology by allowing evaluation of improvement in the case where the endpoint is binary or survival. In the software, the practitioner inputs 1) data from a single-stage randomized trial with one continuous, incidence or survival endpoint and 2) an educated guess of a functional form of a model for the endpoint constructed from domain knowledge. The bootstrap is then employed on data unseen during model fitting to provide confidence intervals for the improvement for the average future patient (assuming future patients are similar to the patients in the trial). One may also test against a null scenario where the hypothesized personalization are not more useful than a standard of care. We demonstrate our method's promise on simulated data as well as on data from a randomized comparative trial investigating two treatments for depression.

Predictive approaches to heterogeneous treatment effects: a scoping review

BACKGROUND

Recent evidence suggests that there is often substantial variation in the benefits and harms across a trial population. We aimed to identify regression modeling approaches that assess heterogeneity of treatment effect within a randomized clinical trial.

METHODS

We performed a literature review using a broad search strategy, complemented by suggestions of a technical expert panel.

RESULTS

The approaches are classified into 3 categories: 1) Risk-based methods (11 papers) use only prognostic factors to define patient subgroups, relying on the mathematical dependency of the absolute risk difference on baseline risk; 2) Treatment effect modeling methods (9 papers) use both prognostic factors and treatment effect modifiers to explore characteristics that interact with the effects of therapy on a relative scale. These methods couple data-driven subgroup identification with approaches to prevent overfitting, such as penalization or use of separate data sets for subgroup identification and effect estimation. 3) Optimal treatment regime methods (12 papers) focus primarily on treatment effect modifiers to classify the trial population into those who benefit from treatment and those who do not. Finally, we also identified papers which describe model evaluation methods (4 papers).

CONCLUSIONS

Three classes of approaches were identified to assess heterogeneity of treatment effect. Methodological research, including both simulations and empirical evaluations, is required to compare the available methods in different settings and to derive well-informed guidance for their application in RCT analysis.

Resampling-based confidence intervals for model-free robust inference on optimal treatment regimes

We propose a new procedure for inference on optimal treatment regimes in the model-free setting, which does not require to specify an outcome regression model. Existing model-free estimators for optimal treatment regimes are usually not suitable for the purpose of inference, because they either have nonstandard asymptotic distributions or do not necessarily guarantee consistent estimation of the parameter indexing the Bayes rule due to the use of surrogate loss. We first study a smoothed robust estimator that directly targets the parameter corresponding to the Bayes decision rule for optimal treatment regimes estimation. This estimator is shown to have an asymptotic normal distribution. Furthermore, we verify that a resampling procedure provides asymptotically accurate inference for both the parameter indexing the optimal treatment regime and the optimal value function. A new algorithm is developed to calculate the proposed estimator with substantially improved speed and stability. Numerical results demonstrate the satisfactory performance of the new methods.

Optimal individualized decision rules using instrumental variable methods

There is an extensive literature on the estimation and evaluation of optimal individualized treatment rules in settings where all confounders of the effect of treatment on outcome are observed. We study the development of individualized decision rules in settings where some of these confounders may not have been measured but a valid binary instrument is available for a binary treatment. We first consider individualized treatment rules, which will naturally be most interesting in settings where it is feasible to intervene directly on treatment. We then consider a setting where intervening on treatment is infeasible, but intervening to encourage treatment is feasible. In both of these settings, we also handle the case that the treatment is a limited resource so that optimal interventions focus the available resources on those individuals who will benefit most from treatment. Given a reference rule, we evaluate an optimal individualized rule by its average causal effect relative to a prespecified reference rule. We develop methods to estimate optimal individualized rules and construct asymptotically efficient plug-in estimators of the corresponding average causal effect relative to a prespecified reference rule.

Estimating the quality of optimal treatment regimes

When multiple treatment alternatives are available for a disease, an obvious question is which alternative is most effective for which patient. One may address this question by searching for optimal treatment regimes that specify for each individual the preferable treatment alternative based on that individual's baseline characteristics. When such a regime has been estimated, its quality (in terms of the expected outcome if it was used for treatment assignment of all patients in the population under study) is of obvious interest. Obtaining a good and reliable estimate of this quantity is a key challenge for which so far no satisfactory solution is available. In this paper, we consider for this purpose several estimators of the expected outcome in conjunction with several resampling methods. The latter have been evaluated before within the context of statistical learning to estimate the prediction error of estimated prediction rules. Yet, the results of these evaluations were equivocal, with different best performing methods in different studies, and with near-zero and even negative correlations between true and estimated prediction errors. Moreover, for different reasons, it is not straightforward to extrapolate the findings of these studies to the context of optimal treatment regimes. To address these issues, we set up a new and comprehensive simulation study. In this study, combinations of different estimators with .632⁺ and out-of-bag bootstrap resampling methods performed best. In addition, the study shed a surprising new light on the previously reported problematic correlations between true and estimated prediction errors in the area of statistical learning.

Precision Medicine

Precision medicine seeks to maximize the quality of healthcare by individualizing the healthcare process to the uniquely evolving health status of each patient. This endeavor spans a broad range of scientific areas including drug discovery, genetics/genomics, health communication, and causal inference all in support of evidence-based, i.e., data-driven, decision making. Precision medicine is formalized as a treatment regime which comprises a sequence of decision rules, one per decision point, which map up-to-date patient information to a recommended action. The potential actions could be the selection of which drug to use, the selection of dose, timing of administration, specific diet or exercise recommendation, or other aspects of treatment or care. Statistics research in precision medicine is broadly focused on methodological development for estimation of and inference for treatment regimes which maximize some cumulative clinical outcome. In this review, we provide an overview of this vibrant area of research and present important and emerging challenges.

Studying treatment-effect heterogeneity in precision medicine through induced subgroups

Precision medicine, in the sense of tailoring the choice of medical treatment to patients' pretreatment characteristics, is nowadays gaining a lot of attention. Preferably, this tailoring should be realized in an evidence-based way, with key evidence in this regard pertaining to subgroups of patients that respond differentially to treatment (i.e., to subgroups involved in treatment-subgroup interactions). Often a-priori hypotheses on subgroups involved in treatment-subgroup interactions are lacking or are incomplete at best. Therefore, methods are needed that can induce such subgroups from empirical data on treatment effectiveness in a post hoc manner. Recently, quite a few such methods have been developed. So far, however, there is little empirical experience in their usage. This may be problematic for medical statisticians and statistically minded medical researchers, as many (nontrivial) choices have to be made during the data-analytic process. The main purpose of this paper is to discuss the major concepts and considerations when using these methods. This discussion will be based on a systematic, conceptual, and technical analysis of the type of research questions at play, and of the type of data that the methods can handle along with the available software, and a review of available empirical evidence. We will illustrate all this with the analysis of a dataset comparing several anti-depressant treatments.

TARGETED SEQUENTIAL DESIGN FOR TARGETED LEARNING INFERENCE OF THE OPTIMAL TREATMENT RULE AND ITS MEAN REWARD

This article studies the targeted sequential inference of an optimal treatment rule (TR) and its mean reward in the non-exceptional case, i.e., assuming that there is no stratum of the baseline covariates where treatment is neither beneficial nor harmful, and under a companion margin assumption. Our pivotal estimator, whose definition hinges on the targeted minimum loss estimation (TMLE) principle, actually infers the mean reward under the current estimate of the optimal TR. This data-adaptive statistical parameter is worthy of interest on its own. Our main result is a central limit theorem which enables the construction of confidence intervals on both mean rewards under the current estimate of the optimal TR and under the optimal TR itself. The asymptotic variance of the estimator takes the form of the variance of an efficient influence curve at a limiting distribution, allowing to discuss the efficiency of inference. As a by product, we also derive confidence intervals on two cumulated pseudo-regrets, a key notion in the study of bandits problems. A simulation study illustrates the procedure. One of the corner-stones of the theoretical study is a new maximal inequality for martingales with respect to the uniform entropy integral.

Evaluating the impact of treating the optimal subgroup

Suppose we have a binary treatment used to influence an outcome. Given data from an observational or controlled study, we wish to determine whether or not there exists some subset of observed covariates in which the treatment is more effective than the standard practice of no treatment. Furthermore, we wish to quantify the improvement in population mean outcome that will be seen if this subgroup receives treatment and the rest of the population remains untreated. We show that this problem is surprisingly challenging given how often it is an (at least implicit) study objective. Blindly applying standard techniques fails to yield any apparent asymptotic results, while using existing techniques to confront the non-regularity does not necessarily help at distributions where there is no treatment effect. Here, we describe an approach to estimate the impact of treating the subgroup which benefits from treatment that is valid in a nonparametric model and is able to deal with the case where there is no treatment effect. The approach is a slight modification of an approach that recently appeared in the individualized medicine literature.

Prediction and tolerance intervals for dynamic treatment regimes

We develop and evaluate tolerance interval methods for dynamic treatment regimes (DTRs) that can provide more detailed prognostic information to patients who will follow an estimated optimal regime. Although the problem of constructing confidence intervals for DTRs has been extensively studied, prediction and tolerance intervals have received little attention. We begin by reviewing in detail different interval estimation and prediction methods and then adapting them to the DTR setting. We illustrate some of the challenges associated with tolerance interval estimation stemming from the fact that we do not typically have data that were generated from the estimated optimal regime. We give an extensive empirical evaluation of the methods and discussed several practical aspects of method choice, and we present an example application using data from a clinical trial. Finally, we discuss future directions within this important emerging area of DTR research.

Optimal Individualized Treatments in Resource-Limited Settings

An individualized treatment rule (ITR) is a treatment rule which assigns treatments to individuals based on (a subset of) their measured covariates. An optimal ITR is the ITR which maximizes the population mean outcome. Previous works in this area have assumed that treatment is an unlimited resource so that the entire population can be treated if this strategy maximizes the population mean outcome. We consider optimal ITRs in settings where the treatment resource is limited so that there is a maximum proportion of the population which can be treated. We give a general closed-form expression for an optimal stochastic ITR in this resource-limited setting, and a closed-form expression for the optimal deterministic ITR under an additional assumption. We also present an estimator of the mean outcome under the optimal stochastic ITR in a large semiparametric model that at most places restrictions on the probability of treatment assignment given covariates. We give conditions under which our estimator is efficient among all regular and asymptotically linear estimators. All of our results are supported by simulations.

STATISTICAL INFERENCE FOR THE MEAN OUTCOME UNDER A POSSIBLY NON-UNIQUE OPTIMAL TREATMENT STRATEGY

We consider challenges that arise in the estimation of the mean outcome under an optimal individualized treatment strategy defined as the treatment rule that maximizes the population mean outcome, where the candidate treatment rules are restricted to depend on baseline covariates. We prove a necessary and sufficient condition for the pathwise differentiability of the optimal value, a key condition needed to develop a regular and asymptotically linear (RAL) estimator of the optimal value. The stated condition is slightly more general than the previous condition implied in the literature. We then describe an approach to obtain root-n rate confidence intervals for the optimal value even when the parameter is not pathwise differentiable. We provide conditions under which our estimator is RAL and asymptotically efficient when the mean outcome is pathwise differentiable. We also outline an extension of our approach to a multiple time point problem. All of our results are supported by simulations.

Using pilot data to size a two-arm randomized trial to find a nearly optimal personalized treatment strategy

A personalized treatment strategy formalizes evidence-based treatment selection by mapping patient information to a recommended treatment. Personalized treatment strategies can produce better patient outcomes while reducing cost and treatment burden. Thus, among clinical and intervention scientists, there is a growing interest in conducting randomized clinical trials when one of the primary aims is estimation of a personalized treatment strategy. However, at present, there are no appropriate sample size formulae to assist in the design of such a trial. Furthermore, because the sampling distribution of the estimated outcome under an estimated optimal treatment strategy can be highly sensitive to small perturbations in the underlying generative model, sample size calculations based on standard (uncorrected) asymptotic approximations or computer simulations may not be reliable. We offer a simple and robust method for powering a single stage, two-armed randomized clinical trial when the primary aim is estimating the optimal single stage personalized treatment strategy. The proposed method is based on inverting a plugin projection confidence interval and is thereby regular and robust to small perturbations of the underlying generative model. The proposed method requires elicitation of two clinically meaningful parameters from clinical scientists and uses data from a small pilot study to estimate nuisance parameters, which are not easily elicited. The method performs well in simulated experiments and is illustrated using data from a pilot study of time to conception and fertility awareness.