1
|
Moodie EEM, Bian Z, Coulombe J, Lian Y, Yang AY, Shortreed SM. Variable selection in high dimensions for discrete-outcome individualized treatment rules: Reducing severity of depression symptoms. Biostatistics 2024; 25:633-647. [PMID: 37660312 DOI: 10.1093/biostatistics/kxad022] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2022] [Revised: 07/14/2023] [Accepted: 08/03/2023] [Indexed: 09/05/2023] Open
Abstract
Despite growing interest in estimating individualized treatment rules, little attention has been given the binary outcome setting. Estimation is challenging with nonlinear link functions, especially when variable selection is needed. We use a new computational approach to solve a recently proposed doubly robust regularized estimating equation to accomplish this difficult task in a case study of depression treatment. We demonstrate an application of this new approach in combination with a weighted and penalized estimating equation to this challenging binary outcome setting. We demonstrate the double robustness of the method and its effectiveness for variable selection. The work is motivated by and applied to an analysis of treatment for unipolar depression using a population of patients treated at Kaiser Permanente Washington.
Collapse
Affiliation(s)
- Erica E M Moodie
- McGill University, Department of Epidemiology & Biostatistics, 2001 McGill College Ave, Suite 1200, Montreal, QC Canada H3A 1G1
| | - Zeyu Bian
- McGill University, Department of Epidemiology & Biostatistics, 2001 McGill College Ave, Suite 1200, Montreal, QC Canada H3A 1G1
| | - Janie Coulombe
- Université de Montréal, Department of Mathematics & Statistics, Pavillon André-Aisenstadt, Montréal, QC Canada H3C 3J7
| | - Yi Lian
- McGill University, Department of Epidemiology & Biostatistics, 2001 McGill College Ave, Suite 1200, Montreal, QC Canada H3A 1G1
| | - Archer Y Yang
- McGill University, Department of Mathematics & Statistics, 805 Sherbrooke Street West Montreal, QC Canada H3A 0B9
| | - Susan M Shortreed
- Kaiser Permanente Washington Health Research Institute, 1730 Minor Ave, Suite 1600, Seattle, WA 98101
- University of Washington, Department of Biostatistics, 1705 NE Pacific St, Seattle, WA 98195
| |
Collapse
|
2
|
Spicker D, Moodie EE, Shortreed SM. Differentially Private Outcome-Weighted Learning for Optimal Dynamic Treatment Regime Estimation. Stat (Int Stat Inst) 2024; 13:e641. [PMID: 39070170 PMCID: PMC11281278 DOI: 10.1002/sta4.641] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2023] [Accepted: 11/12/2023] [Indexed: 07/30/2024]
Abstract
Precision medicine is a framework for developing evidence-based medical recommendations that seeks to determine the optimal sequence of treatments tailored to all of the relevant patient-level characteristics which are observable. Because precision medicine relies on highly sensitive, patient-level data, ensuring the privacy of participants is of great importance. Dynamic treatment regimes (DTRs) provide one formalization of precision medicine in a longitudinal setting. Outcome-Weighted Learning (OWL) is a family of techniques for estimating optimal DTRs based on observational data. OWL techniques leverage support vector machine (SVM) classifiers in order to perform estimation. SVMs perform classification based on a set of influential points in the data known as support vectors. The classification rule produced by SVMs often requires direct access to the support vectors. Thus, releasing a treatment policy estimated with OWL requires the release of patient data for a subset of patients in the sample. As a result, the classification rules from SVMs constitute a severe privacy violation for those individuals whose data comprise the support vectors. This privacy violation is a major concern, particularly in light of the potentially highly sensitive medical data which are used in DTR estimation. Differential privacy has emerged as a mathematical framework for ensuring the privacy of individual-level data, with provable guarantees on the likelihood that individual characteristics can be determined by an adversary. We provide the first investigation of differential privacy in the context of DTRs and provide a differentially private OWL estimator, with theoretical results allowing us to quantify the cost of privacy in terms of the accuracy of the private estimators.
Collapse
Affiliation(s)
- Dylan Spicker
- Department of Mathematics and Statistics, University of New Brunswick (Saint John), NB, Canada
| | - Erica E.M. Moodie
- Department of Epidemiology, Biostatistics, and Occupational Health, McGill University, QC, Canada
| | - Susan M. Shortreed
- Kaiser Permanente Washington Health Research Institute, WA, USA
- Department of Biostatistics University of Washington, WA, USA
| |
Collapse
|
3
|
Shah KS, Fu H, Kosorok MR. Stabilized direct learning for efficient estimation of individualized treatment rules. Biometrics 2023; 79:2843-2856. [PMID: 36585916 DOI: 10.1111/biom.13818] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2021] [Accepted: 12/23/2022] [Indexed: 01/01/2023]
Abstract
In recent years, the field of precision medicine has seen many advancements. Significant focus has been placed on creating algorithms to estimate individualized treatment rules (ITRs), which map from patient covariates to the space of available treatments with the goal of maximizing patient outcome. Direct learning (D-Learning) is a recent one-step method which estimates the ITR by directly modeling the treatment-covariate interaction. However, when the variance of the outcome is heterogeneous with respect to treatment and covariates, D-Learning does not leverage this structure. Stabilized direct learning (SD-Learning), proposed in this paper, utilizes potential heteroscedasticity in the error term through a residual reweighting which models the residual variance via flexible machine learning algorithms such as XGBoost and random forests. We also develop an internal cross-validation scheme which determines the best residual model among competing models. SD-Learning improves the efficiency of D-Learning estimates in binary and multi-arm treatment scenarios. The method is simple to implement and an easy way to improve existing algorithms within the D-Learning family, including original D-Learning, Angle-based D-Learning (AD-Learning), and Robust D-learning (RD-Learning). We provide theoretical properties and justification of the optimality of SD-Learning. Head-to-head performance comparisons with D-Learning methods are provided through simulations, which demonstrate improvement in terms of average prediction error (APE), misclassification rate, and empirical value, along with a data analysis of an acquired immunodeficiency syndrome (AIDS) randomized clinical trial.
Collapse
Affiliation(s)
- Kushal S Shah
- Department of Biostatistics, University of North Carolina at Chapel Hill, North Carolina, USA
| | - Haoda Fu
- Eli Lilly and Company, Lilly Corporate Center, Indianapolis, USA
| | - Michael R Kosorok
- Department of Biostatistics, University of North Carolina at Chapel Hill, North Carolina, USA
| |
Collapse
|
4
|
Li Z, Chen J, Laber E, Liu F, Baumgartner R. Optimal Treatment Regimes: A Review and Empirical Comparison. Int Stat Rev 2023. [DOI: 10.1111/insr.12536] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/24/2023]
Affiliation(s)
- Zhen Li
- Department of Statistics North Carolina State University Raleigh 27607 NC USA
| | - Jie Chen
- Department of Biometrics Overland Pharmaceuticals Dover 19901 DE USA
| | - Eric Laber
- Department of Statistical Science, Department of Biostatistics and Bioinformatics Duke University Durham 27708 NC USA
| | - Fang Liu
- Biostatistics and Research Decision Sciences Merck & Co., Inc. Kenilworth NJ 07033 USA
| | - Richard Baumgartner
- Biostatistics and Research Decision Sciences Merck & Co., Inc. Kenilworth NJ 07033 USA
| |
Collapse
|
5
|
Accountable survival contrast-learning for optimal dynamic treatment regimes. Sci Rep 2023; 13:2250. [PMID: 36755137 PMCID: PMC9908913 DOI: 10.1038/s41598-023-29106-w] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/12/2022] [Accepted: 01/30/2023] [Indexed: 02/10/2023] Open
Abstract
Dynamic treatment regime (DTR) is an emerging paradigm in recent medical studies, which searches a series of decision rules to assign optimal treatments to each patient by taking into account individual features such as genetic, environmental, and social factors. Although there is a large and growing literature on statistical methods to estimate optimal treatment regimes, most methodologies focused on complete data. In this article, we propose an accountable contrast-learning algorithm for optimal dynamic treatment regime with survival endpoints. Our estimating procedure is originated from a doubly-robust weighted classification scheme, which is a model-based contrast-learning method that directly characterizes the interaction terms between predictors and treatments without main effects. To reflect the censorship, we adopt the pseudo-value approach that replaces survival quantities with pseudo-observations for the time-to-event outcome. Unlike many existing approaches, mostly based on complicated outcome regression modeling or inverse-probability weighting schemes, the pseudo-value approach greatly simplifies the estimating procedure for optimal treatment regime by allowing investigators to conveniently apply standard machine learning techniques to censored survival data without losing much efficiency. We further explore a SCAD-penalization to find informative clinical variables and modified algorithms to handle multiple treatment options by searching upper and lower bounds of the objective function. We demonstrate the utility of our proposal via extensive simulations and application to AIDS data.
Collapse
|
6
|
Zhou J, Zhang Y, Tu W. A reference-free R-learner for treatment recommendation. Stat Methods Med Res 2023; 32:404-424. [PMID: 36540907 DOI: 10.1177/09622802221144326] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/24/2022]
Abstract
Assigning optimal treatments to individual patients based on their characteristics is the ultimate goal of precision medicine. Deriving evidence-based recommendations from observational data while considering the causal treatment effects and patient heterogeneity is a challenging task, especially in situations of multiple treatment options. Herein, we propose a reference-free R-learner based on a simplex algorithm for treatment recommendation. We showed through extensive simulation that the proposed method produced accurate recommendations that corresponded to optimal treatment outcomes, regardless of the reference group. We used the method to analyze data from the Systolic Blood Pressure Intervention Trial (SPRINT) and achieved recommendations consistent with the current clinical guidelines.
Collapse
Affiliation(s)
- Junyi Zhou
- Design and Inovation, 7129Amgen Inc., Thousand Oaks, CA, USA
| | - Ying Zhang
- Department of Biostatistics, 12284University of Nebraska Medical Center, Omaha, NE, USA
| | - Wanzhu Tu
- Department of Biostatistics and Health Data Science, Indiana University-School of Medicine and Fairbanks School of Public Health, Indianapolis, IN, USA
| |
Collapse
|
7
|
Oh EJ, Qian M, Cheung YK. Generalization error bounds of dynamic treatment regimes in penalized regression-based learning. Ann Stat 2022. [DOI: 10.1214/22-aos2171] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
Affiliation(s)
- Eun Jeong Oh
- Department of Biostatistics, Columbia University
| | - Min Qian
- Department of Biostatistics, Columbia University
| | | |
Collapse
|
8
|
Zhang B, Zhang M. Subgroup identification and variable selection for treatment decision making. Ann Appl Stat 2022. [DOI: 10.1214/21-aoas1468] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
Affiliation(s)
- Baqun Zhang
- School of Statistics and Management, Shanghai University of Finance and Economics
| | - Min Zhang
- Department of Biostatistics, University of Michigan
| |
Collapse
|
9
|
Survival Augmented Patient Preference Incorporated Reinforcement Learning to Evaluate Tailoring Variables for Personalized Healthcare. STATS 2021. [DOI: 10.3390/stats4040046] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022] Open
Abstract
In this paper, we consider personalized treatment decision strategies in the management of chronic diseases, such as chronic kidney disease, which typically consists of sequential and adaptive treatment decision making. We investigate a two-stage treatment setting with a survival outcome that could be right censored. This can be formulated through a dynamic treatment regime (DTR) framework, where the goal is to tailor treatment to each individual based on their own medical history in order to maximize a desirable health outcome. We develop a new method, Survival Augmented Patient Preference incorporated reinforcement Q-Learning (SAPP-Q-Learning) to decide between quality of life and survival restricted at maximal follow-up. Our method incorporates the latent patient preference into a weighted utility function that balances between quality of life and survival time, in a Q-learning model framework. We further propose a corresponding m-out-of-n Bootstrap procedure to accurately make statistical inferences and construct confidence intervals on the effects of tailoring variables, whose values can guide personalized treatment strategies.
Collapse
|
10
|
Guo W, Zhou XH, Ma S. Estimation of Optimal Individualized Treatment Rules Using a Covariate-Specific Treatment Effect Curve With High-Dimensional Covariates. J Am Stat Assoc 2021. [DOI: 10.1080/01621459.2020.1865167] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/16/2022]
Affiliation(s)
- Wenchuan Guo
- Department of Statistics, University of California Riverside, Riverside, CA
- Global Biometric Sciences, Bristol-Myers Squibb, Pennington, NJ
| | - Xiao-Hua Zhou
- Beijing International Center for Mathematical Research, and Department of Biostatistics, Peking University, Beijing, China
| | - Shujie Ma
- Department of Statistics, University of California Riverside, Riverside, CA
| |
Collapse
|
11
|
Dong L, Laber E, Goldberg Y, Song R, Yang S. Ascertaining properties of weighting in the estimation of optimal treatment regimes under monotone missingness. Stat Med 2020; 39:3503-3520. [PMID: 32729973 DOI: 10.1002/sim.8678] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2019] [Revised: 04/28/2020] [Accepted: 04/30/2020] [Indexed: 11/10/2022]
Abstract
Dynamic treatment regimes operationalize precision medicine as a sequence of decision rules, one per stage of clinical intervention, that map up-to-date patient information to a recommended intervention. An optimal treatment regime maximizes the mean utility when applied to the population of interest. Methods for estimating an optimal treatment regime assume the data to be fully observed, which rarely occurs in practice. A common approach is to first use multiple imputation and then pool the estimators across imputed datasets. However, this approach requires estimating the joint distribution of patient trajectories, which can be high-dimensional, especially when there are multiple stages of intervention. We examine the application of inverse probability weighted estimating equations as an alternative to multiple imputation in the context of monotonic missingness. This approach applies to a broad class of estimators of an optimal treatment regime including both Q-learning and a generalization of outcome weighted learning. We establish consistency under mild regularity conditions and demonstrate its advantages in finite samples using a series of simulation experiments and an application to a schizophrenia study.
Collapse
Affiliation(s)
- Lin Dong
- Department of Statistics, North Carolina State University, Raleigh, North Carolina, USA
| | - Eric Laber
- Department of Statistics, North Carolina State University, Raleigh, North Carolina, USA
| | - Yair Goldberg
- Department of Statistics, Technion Israel Institute of Technology, Haifa, Israel
| | - Rui Song
- Department of Statistics, North Carolina State University, Raleigh, North Carolina, USA
| | - Shu Yang
- Department of Statistics, North Carolina State University, Raleigh, North Carolina, USA
| |
Collapse
|
12
|
Affiliation(s)
- Yilun Sun
- Department of Biostatistics, University of Michigan, Ann Arbor, MI
- Department of Radiation Oncology, University of Michigan, Ann Arbor, MI
| | - Lu Wang
- Department of Biostatistics, University of Michigan, Ann Arbor, MI
| |
Collapse
|
13
|
Cui Y, Tchetgen ET. A semiparametric instrumental variable approach to optimal treatment regimes under endogeneity. J Am Stat Assoc 2020; 116:162-173. [PMID: 33994604 PMCID: PMC8118566 DOI: 10.1080/01621459.2020.1783272] [Citation(s) in RCA: 17] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2019] [Revised: 02/05/2020] [Accepted: 06/09/2020] [Indexed: 01/23/2023]
Abstract
There is a fast-growing literature on estimating optimal treatment regimes based on randomized trials or observational studies under a key identifying condition of no unmeasured confounding. Because confounding by unmeasured factors cannot generally be ruled out with certainty in observational studies or randomized trials subject to noncompliance, we propose a general instrumental variable approach to learning optimal treatment regimes under endogeneity. Specifically, we establish identification of both value function E [ Y D ( L ) ] for a given regime D and optimal regimes arg max D E [ Y D ( L ) ] with the aid of a binary instrumental variable, when no unmeasured confounding fails to hold. We also construct novel multiply robust classification-based estimators. Furthermore, we propose to identify and estimate optimal treatment regimes among those who would comply to the assigned treatment under a monotonicity assumption. In this latter case, we establish the somewhat surprising result that complier optimal regimes can be consistently estimated without directly collecting compliance information and therefore without the complier average treatment effect itself being identified. Our approach is illustrated via extensive simulation studies and a data application on the effect of child rearing on labor participation.
Collapse
Affiliation(s)
- Yifan Cui
- Department of Statistics, The Wharton School, University of Pennsylvania, Philadelphia, PA 19104
| | - Eric Tchetgen Tchetgen
- Department of Statistics, The Wharton School, University of Pennsylvania, Philadelphia, PA 19104
| |
Collapse
|
14
|
Liu Y, Wang Y, Kosorok MR, Zhao Y, Zeng D. Augmented outcome-weighted learning for estimating optimal dynamic treatment regimens. Stat Med 2018; 37:3776-3788. [PMID: 29873099 DOI: 10.1002/sim.7844] [Citation(s) in RCA: 36] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2017] [Revised: 03/30/2018] [Accepted: 05/12/2018] [Indexed: 11/08/2022]
Abstract
Dynamic treatment regimens (DTRs) are sequential treatment decisions tailored by patient's evolving features and intermediate outcomes at each treatment stage. Patient heterogeneity and the complexity and chronicity of many diseases call for learning optimal DTRs that can best tailor treatment according to each individual's time-varying characteristics (eg, intermediate response over time). In this paper, we propose a robust and efficient approach referred to as Augmented Outcome-weighted Learning (AOL) to identify optimal DTRs from sequential multiple assignment randomized trials. We improve previously proposed outcome-weighted learning to allow for negative weights. Furthermore, to reduce the variability of weights for numeric stability and improve estimation accuracy, in AOL, we propose a robust augmentation to the weights by making use of predicted pseudooutcomes from regression models for Q-functions. We show that AOL still yields Fisher-consistent DTRs even if the regression models are misspecified and that an appropriate choice of the augmentation guarantees smaller stochastic errors in value function estimation for AOL than the previous outcome-weighted learning. Finally, we establish the convergence rates for AOL. The comparative advantage of AOL over existing methods is demonstrated through extensive simulation studies and an application to a sequential multiple assignment randomized trial for major depressive disorder.
Collapse
Affiliation(s)
- Ying Liu
- Division of Biostatistics, Medical College of Wisconsin, Milwaukee, WI, USA
| | - Yuanjia Wang
- Department of Biostatistics, Columbia University, New York City, NY, USA
| | - Michael R Kosorok
- Department of Biostatistics, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
| | - Yingqi Zhao
- Public Health Division, Fred Hutchinson Cancer Research Center, Seattle, WA, USA
| | - Donglin Zeng
- Department of Biostatistics, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
| |
Collapse
|