1
|
Li Z, Chen J, Laber E, Liu F, Baumgartner R. Optimal Treatment Regimes: A Review and Empirical Comparison. Int Stat Rev 2023. [DOI: 10.1111/insr.12536] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/24/2023]
Affiliation(s)
- Zhen Li
- Department of Statistics North Carolina State University Raleigh 27607 NC USA
| | - Jie Chen
- Department of Biometrics Overland Pharmaceuticals Dover 19901 DE USA
| | - Eric Laber
- Department of Statistical Science, Department of Biostatistics and Bioinformatics Duke University Durham 27708 NC USA
| | - Fang Liu
- Biostatistics and Research Decision Sciences Merck & Co., Inc. Kenilworth NJ 07033 USA
| | - Richard Baumgartner
- Biostatistics and Research Decision Sciences Merck & Co., Inc. Kenilworth NJ 07033 USA
| |
Collapse
|
2
|
Zhou J, Zhang Y, Tu W. A reference-free R-learner for treatment recommendation. Stat Methods Med Res 2023; 32:404-424. [PMID: 36540907 PMCID: PMC11376343 DOI: 10.1177/09622802221144326] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/24/2022]
Abstract
Assigning optimal treatments to individual patients based on their characteristics is the ultimate goal of precision medicine. Deriving evidence-based recommendations from observational data while considering the causal treatment effects and patient heterogeneity is a challenging task, especially in situations of multiple treatment options. Herein, we propose a reference-free R-learner based on a simplex algorithm for treatment recommendation. We showed through extensive simulation that the proposed method produced accurate recommendations that corresponded to optimal treatment outcomes, regardless of the reference group. We used the method to analyze data from the Systolic Blood Pressure Intervention Trial (SPRINT) and achieved recommendations consistent with the current clinical guidelines.
Collapse
Affiliation(s)
- Junyi Zhou
- Design and Inovation, 7129Amgen Inc., Thousand Oaks, CA, USA
| | - Ying Zhang
- Department of Biostatistics, 12284University of Nebraska Medical Center, Omaha, NE, USA
| | - Wanzhu Tu
- Department of Biostatistics and Health Data Science, Indiana University-School of Medicine and Fairbanks School of Public Health, Indianapolis, IN, USA
| |
Collapse
|
3
|
Wang J, Marion-Gallois R. Propensity score matching and stratification using multiparty data without pooling. Pharm Stat 2023; 22:4-19. [PMID: 35733398 DOI: 10.1002/pst.2250] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/26/2021] [Revised: 04/01/2022] [Accepted: 05/29/2022] [Indexed: 02/01/2023]
Abstract
Matching and stratification based on confounding factors or propensity scores (PS) are powerful approaches for reducing confounding bias in indirect treatment comparisons. However, implementing these approaches requires pooled individual patient data (IPD). The research presented here was motivated by an indirect comparison between a single-armed trial in acute myeloid leukemia (AML), and two external AML registries with current treatments for a control. For confidentiality reasons, IPD cannot be pooled. Common approaches to adjusting confounding bias, such as PS matching or stratification, cannot be applied as 1) a model for PS, for example, a logistic model, cannot be fitted without pooling covariate data; 2) pooling response data may be necessary for some statistical inference (e.g., estimating the SE of mean difference of matched pairs) after PS matching. We propose a set of approaches that do not require pooling IPD, using a combination of methods including a linear discriminant for matching and stratification, and secure multiparty computation for estimation of within-pair sample variance and for calculations involving multiple control sources. The approaches only need to share aggregated data offline, rather than real-time secure data transfer, as required by typical secure multiparty computation for model fitting. For survival analysis, we propose an approach using restricted mean survival time. A simulation study was conducted to evaluate this approach in several scenarios, in particular, with a mixture of continuous and binary covariates. The results confirmed the robustness and efficiency of the proposed approach. A real data example is also provided for illustration.
Collapse
|
4
|
Cheng JJ, Huling JD, Chen G. Meta-analysis of individualized treatment rules via sign-coherency. PROCEEDINGS OF MACHINE LEARNING RESEARCH 2022; 193:171-198. [PMID: 37786410 PMCID: PMC10544849] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Subscribe] [Scholar Register] [Indexed: 10/04/2023]
Abstract
Medical treatments tailored to a patient's baseline characteristics hold the potential of improving patient outcomes while reducing negative side effects. Learning individualized treatment rules (ITRs) often requires aggregation of multiple datasets(sites); however, current ITR methodology does not take between-site heterogeneity into account, which can hurt model generalizability when deploying back to each site. To address this problem, we develop a method for individual-level meta-analysis of ITRs, which jointly learns site-specific ITRs while borrowing information about feature sign-coherency via a scientifically-motivated directionality principle. We also develop an adaptive procedure for model tuning, using information criteria tailored to the ITR learning problem. We study the proposed methods through numerical experiments to understand their performance under different levels of between-site heterogeneity and apply the methodology to estimate ITRs in a large multi-center database of electronic health records. This work extends several popular methodologies for estimating ITRs (A-learning, weighted learning) to the multiple-sites setting.
Collapse
Affiliation(s)
- Jay Jojo Cheng
- Department of Biostatistics and Medical Informatics, University of Wisconsin-Madison
| | | | - Guanhua Chen
- Department of Biostatistics and Medical Informatics, University of Wisconsin-Madison
| |
Collapse
|
5
|
Shi C, Luo S, Le Y, Zhu H, Song R. Statistically Efficient Advantage Learning for Offline Reinforcement Learning in Infinite Horizons. J Am Stat Assoc 2022. [DOI: 10.1080/01621459.2022.2106868] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/16/2022]
Affiliation(s)
| | | | - Yuan Le
- Shanghai University of Finance and Economics
| | - Hongtu Zhu
- University of North Carolina at Chapel Hill
| | | |
Collapse
|
6
|
Chen V, Li C, Zhang H. dipm: an R package implementing the Depth Importance in Precision Medicine (DIPM) tree and Forest-based method. BIOINFORMATICS ADVANCES 2022; 2:vbac041. [PMID: 35785020 PMCID: PMC9245626 DOI: 10.1093/bioadv/vbac041] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/21/2022] [Revised: 05/19/2022] [Accepted: 06/09/2022] [Indexed: 01/27/2023]
Abstract
Summary The Depth Importance in Precision Medicine (DIPM) method is a classification tree designed for the identification of subgroups relevant to the precision medicine setting. In this setting, a relevant subgroup is a subgroup in which subjects perform either especially well or poorly with a particular treatment assignment. Herein, we introduce, dipm, a novel R package that implements the DIPM method using R code that calls a program in C. Availability and implementation dipm is available under a GPL-3 licence on CRAN https://cran.r-project.org/web/packages/dipm/index.html and at https://ysph.yale.edu/c2s2/software/dipm. It is continuously being developed at https://github.com/chenvict/dipm. Supplementary information Supplementary data are available at Bioinformatics Advances online.
Collapse
|
7
|
Zhou Y, Wang L, Song R, Zhao T. Transformation-Invariant Learning of Optimal Individualized Decision Rules with Time-to-Event Outcomes. J Am Stat Assoc 2022. [DOI: 10.1080/01621459.2022.2068420] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
Affiliation(s)
- Yu Zhou
- Roku, San Jose, United States
| | - Lan Wang
- Department of Management Science, University of Miami
| | - Rui Song
- Department of Statistics, North Carolina State University
| | - Tuoyi Zhao
- Department of Management Science, University of Miami
| |
Collapse
|
8
|
Wu J, Galanter N, Shortreed SM, Moodie EEM. Ranking tailoring variables for constructing individualized treatment rules: an application to schizophrenia. J R Stat Soc Ser C Appl Stat 2022; 71:309-330. [PMID: 38288004 PMCID: PMC10823524 DOI: 10.1111/rssc.12533] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
Abstract
As with many chronic conditions, matching patients with schizophrenia to the best treatment options is difficult. Selecting antipsychotic medication is especially challenging because many of the medications can have burdensome side effects. Adjusting or tailoring medications based on patients' characteristics could improve symptoms. However, it is often not known which patient characteristics are most helpful for informing treatment selection. In this paper, we address the challenge of identifying and ranking important variables for tailoring treatment decisions. We consider a value-search approach implemented through dynamic marginal structural models to estimate an optimal individualized treatment rule. We apply our methodology to the Clinical Antipsychotics Trial of Intervention and Effectiveness (CATIE) study for schizophrenia, to evaluate if some tailoring variables have greater potential than others for selecting treatments for patients with schizophrenia (Stroup et al., 2003).
Collapse
Affiliation(s)
| | | | - Susan M Shortreed
- Kaiser Permanente Washington Health Research Institute, Seattle, USA, and University of Washington, Seattle, USA
| | | |
Collapse
|
9
|
Guo H, Li J, Liu H, He J. Learning dynamic treatment strategies for coronary heart diseases by artificial intelligence: real-world data-driven study. BMC Med Inform Decis Mak 2022; 22:39. [PMID: 35168623 PMCID: PMC8845235 DOI: 10.1186/s12911-022-01774-0] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2021] [Accepted: 02/01/2022] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Coronary heart disease (CHD) has become the leading cause of death and one of the most serious epidemic diseases worldwide. CHD is characterized by urgency, danger and severity, and dynamic treatment strategies for CHD patients are needed. We aimed to build and validate an AI model for dynamic treatment recommendations for CHD patients with the goal of improving patient outcomes and learning best practices from clinicians to help clinical decision support for treating CHD patients. METHODS We formed the treatment strategy as a sequential decision problem, and applied an AI supervised reinforcement learning-long short-term memory (SRL-LSTM) framework that combined supervised learning (SL) and reinforcement learning (RL) with an LSTM network to track patients' states to learn a recommendation model that took a patient's diagnosis and evolving health status as input and provided a treatment recommendation in the form of whether to take specific drugs. The experiments were conducted by leveraging a real-world intensive care unit (ICU) database with 13,762 admitted patients diagnosed with CHD. We compared the performance of the applied SRL-LSTM model and several state-of-the-art SL and RL models in reducing the estimated in-hospital mortality and the Jaccard similarity with clinicians' decisions. We used a random forest algorithm to calculate the feature importance of both the clinician policy and the AI policy to illustrate the interpretability of the AI model. RESULTS Our experimental study demonstrated that the AI model could help reduce the estimated in-hospital mortality through its RL function and learn the best practice from clinicians through its SL function. The similarity between the clinician policy and the AI policy regarding the surviving patients was high, while for the expired patients, it was much lower. The dynamic treatment strategies made by the AI model were clinically interpretable and relied on sensible clinical features extracted according to monitoring indexes and risk factors for CHD patients. CONCLUSIONS We proposed a pipeline for constructing an AI model to learn dynamic treatment strategies for CHD patients that could improve patient outcomes and mimic the best practices of clinicians. And a lot of further studies and efforts are needed to make it practical.
Collapse
Affiliation(s)
- Haihong Guo
- School of Information, Renmin University of China, 59 Zhongguancun Street, Haidian District, Beijing, 100872, China
- Institute of Medical Information/Medical Library, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
- Key Laboratory of Data Engineering and Knowledge Engineering, Ministry of Education, Beijing, China
| | - Jiao Li
- Institute of Medical Information/Medical Library, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Hongyan Liu
- School of Economics and Management, Tsinghua University, Beijing, China
| | - Jun He
- School of Information, Renmin University of China, 59 Zhongguancun Street, Haidian District, Beijing, 100872, China.
- Key Laboratory of Data Engineering and Knowledge Engineering, Ministry of Education, Beijing, China.
| |
Collapse
|
10
|
Shi C, Wang X, Luo S, Zhu H, Ye J, Song R. Dynamic Causal Effects Evaluation in A/B Testing with a Reinforcement Learning Framework. J Am Stat Assoc 2022. [DOI: 10.1080/01621459.2022.2027776] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
Affiliation(s)
| | - Xiaoyu Wang
- Key Laboratory of Systems and Control, Academy of Mathematics and Systems Science, Chinese Academy of Sciences
| | | | - Hongtu Zhu
- The Univeristy of North Carolina at Chapell Hill
| | | | | |
Collapse
|
11
|
Zhou W, Zhu R, Zeng D. A parsimonious personalized dose-finding model via dimension reduction. Biometrika 2021; 108:643-659. [PMID: 34658383 PMCID: PMC8514170 DOI: 10.1093/biomet/asaa087] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022] Open
Abstract
Learning an individualized dose rule in personalized medicine is a challenging statistical problem. Existing methods often suffer from the curse of dimensionality, especially when the decision function is estimated nonparametrically. To tackle this problem, we propose a dimension reduction framework that effectively reduces the estimation to a lower-dimensional subspace of the covariates. We exploit that the individualized dose rule can be defined in a subspace spanned by a few linear combinations of the covariates, leading to a more parsimonious model. Also, our framework does not require the inverse probability of the propensity score under observational studies due to a direct maximization of the value function. This distinguishes us from the outcome weighted learning framework, which also solves decision rules directly. Under the same framework, we further propose a pseudo-direct learning approach focuses more on estimating the dimensionality-reduced subspace of the treatment outcome. Parameters in both approaches can be estimated efficiently using an orthogonality constrained optimization algorithm on the Stiefel manifold. Under mild regularity assumptions, the asymptotic normality results of the proposed estimators can are established, respectively. We also derive the consistency and convergence rate for the value function under the estimated optimal dose rule. We evaluate the performance of the proposed approaches through extensive simulation studies and a warfarin pharmacogenetic dataset.
Collapse
Affiliation(s)
- Wenzhuo Zhou
- Department of Statistics, University of Illinois at Urbana-Champaign, Champaign, Illinois 61820, U.S.A
| | - Ruoqing Zhu
- Department of Statistics, University of Illinois at Urbana-Champaign, Champaign, Illinois 61820, U.S.A
| | - Donglin Zeng
- Department of Biostatistics, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina 27599, U.S.A
| |
Collapse
|
12
|
Guo B, Holscher HD, Auvil LS, Welge ME, Bushell CB, Novotny JA, Baer DJ, Burd NA, Khan NA, Zhu R. Estimating Heterogeneous Treatment Effect on Multivariate Responses Using Random Forests. STATISTICS IN BIOSCIENCES 2021. [DOI: 10.1007/s12561-021-09310-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
|
13
|
Zhang P, Ma J, Chen X, Shentu Y. A nonparametric method for value function guided subgroup identification via gradient tree boosting for censored survival data. Stat Med 2020; 39:4133-4146. [PMID: 32786155 DOI: 10.1002/sim.8714] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2020] [Revised: 06/08/2020] [Accepted: 07/09/2020] [Indexed: 11/07/2022]
Abstract
In randomized clinical trials with survival outcome, there has been an increasing interest in subgroup identification based on baseline genomic, proteomic markers, or clinical characteristics. Some of the existing methods identify subgroups that benefit substantially from the experimental treatment by directly modeling outcomes or treatment effect. When the goal is to find an optimal treatment for a given patient rather than finding the right patient for a given treatment, methods under the individualized treatment regime framework estimate an individualized treatment rule that would lead to the best expected clinical outcome as measured by a value function. Connecting the concept of value function to subgroup identification, we propose a nonparametric method that searches for subgroup membership scores by maximizing a value function that directly reflects the subgroup-treatment interaction effect based on restricted mean survival time. A gradient tree boosting algorithm is proposed to search for the individual subgroup membership scores. We conduct simulation studies to evaluate the performance of the proposed method and an application to an AIDS clinical trial is performed for illustration.
Collapse
Affiliation(s)
- Pingye Zhang
- Biostatistics and Research Decision Sciences, MRL, Merck & Co., Inc., Rahway, New Jersey, USA
| | - Junshui Ma
- Biostatistics and Research Decision Sciences, MRL, Merck & Co., Inc., Rahway, New Jersey, USA
| | - Xinqun Chen
- Biostatistics and Research Decision Sciences, MRL, Merck & Co., Inc., Rahway, New Jersey, USA
| | - Yue Shentu
- Biostatistics and Research Decision Sciences, MRL, Merck & Co., Inc., Rahway, New Jersey, USA
| |
Collapse
|
14
|
Wu Y, Wang L. Resampling-based confidence intervals for model-free robust inference on optimal treatment regimes. Biometrics 2020; 77:465-476. [PMID: 32687215 DOI: 10.1111/biom.13337] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/20/2019] [Accepted: 06/24/2020] [Indexed: 12/01/2022]
Abstract
We propose a new procedure for inference on optimal treatment regimes in the model-free setting, which does not require to specify an outcome regression model. Existing model-free estimators for optimal treatment regimes are usually not suitable for the purpose of inference, because they either have nonstandard asymptotic distributions or do not necessarily guarantee consistent estimation of the parameter indexing the Bayes rule due to the use of surrogate loss. We first study a smoothed robust estimator that directly targets the parameter corresponding to the Bayes decision rule for optimal treatment regimes estimation. This estimator is shown to have an asymptotic normal distribution. Furthermore, we verify that a resampling procedure provides asymptotically accurate inference for both the parameter indexing the optimal treatment regime and the optimal value function. A new algorithm is developed to calculate the proposed estimator with substantially improved speed and stability. Numerical results demonstrate the satisfactory performance of the new methods.
Collapse
Affiliation(s)
- Yunan Wu
- School of Statistics, University of Minnesota, Minneapolis, Minnesota
| | - Lan Wang
- Department of Management Science, University of Miami, Coral Gables, Florida
| |
Collapse
|
15
|
Cui Y, Tchetgen ET. A semiparametric instrumental variable approach to optimal treatment regimes under endogeneity. J Am Stat Assoc 2020; 116:162-173. [PMID: 33994604 PMCID: PMC8118566 DOI: 10.1080/01621459.2020.1783272] [Citation(s) in RCA: 17] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2019] [Revised: 02/05/2020] [Accepted: 06/09/2020] [Indexed: 01/23/2023]
Abstract
There is a fast-growing literature on estimating optimal treatment regimes based on randomized trials or observational studies under a key identifying condition of no unmeasured confounding. Because confounding by unmeasured factors cannot generally be ruled out with certainty in observational studies or randomized trials subject to noncompliance, we propose a general instrumental variable approach to learning optimal treatment regimes under endogeneity. Specifically, we establish identification of both value function E [ Y D ( L ) ] for a given regime D and optimal regimes arg max D E [ Y D ( L ) ] with the aid of a binary instrumental variable, when no unmeasured confounding fails to hold. We also construct novel multiply robust classification-based estimators. Furthermore, we propose to identify and estimate optimal treatment regimes among those who would comply to the assigned treatment under a monotonicity assumption. In this latter case, we establish the somewhat surprising result that complier optimal regimes can be consistently estimated without directly collecting compliance information and therefore without the complier average treatment effect itself being identified. Our approach is illustrated via extensive simulation studies and a data application on the effect of child rearing on labor participation.
Collapse
Affiliation(s)
- Yifan Cui
- Department of Statistics, The Wharton School, University of Pennsylvania, Philadelphia, PA 19104
| | - Eric Tchetgen Tchetgen
- Department of Statistics, The Wharton School, University of Pennsylvania, Philadelphia, PA 19104
| |
Collapse
|
16
|
Chen V, Zhang H. Depth importance in precision medicine (DIPM): a tree- and forest-based method for right-censored survival outcomes. Biostatistics 2020; 23:157-172. [PMID: 32424406 DOI: 10.1093/biostatistics/kxaa021] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/16/2019] [Revised: 04/09/2020] [Accepted: 04/13/2020] [Indexed: 12/26/2022] Open
Abstract
Many clinical trials have been conducted to compare right-censored survival outcomes between interventions. Such comparisons are typically made on the basis of the entire group receiving one intervention versus the others. In order to identify subgroups for which the preferential treatment may differ from the overall group, we propose the depth importance in precision medicine (DIPM) method for such data within the precision medicine framework. The approach first modifies the split criteria of the traditional classification tree to fit the precision medicine setting. Then, a random forest of trees is constructed at each node. The forest is used to calculate depth variable importance scores for each candidate split variable. The variable with the highest score is identified as the best variable to split the node. The importance score is a flexible and simply constructed measure that makes use of the observation that more important variables tend to be selected closer to the root nodes of trees. The DIPM method is primarily designed for the analysis of clinical data with two treatment groups. We also present the extension to the case of more than two treatment groups. We use simulation studies to demonstrate the accuracy of our method and provide the results of applications to two real-world data sets. In the case of one data set, the DIPM method outperforms an existing method, and a primary motivation of this article is the ability of the DIPM method to address the shortcomings of this existing method. Altogether, the DIPM method yields promising results that demonstrate its capacity to guide personalized treatment decisions in cases with right-censored survival outcomes.
Collapse
Affiliation(s)
- Victoria Chen
- Department of Biostatistics, Yale School of Public Health, New Haven, CT, USA
| | - Heping Zhang
- Department of Biostatistics, Yale School of Public Health, New Haven, CT, USA
| |
Collapse
|
17
|
Zhao YQ, Zhu R, Chen G, Zheng Y. Constructing dynamic treatment regimes with shared parameters for censored data. Stat Med 2020; 39:1250-1263. [PMID: 31951041 PMCID: PMC7305816 DOI: 10.1002/sim.8473] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/08/2019] [Revised: 10/31/2019] [Accepted: 12/16/2019] [Indexed: 01/28/2023]
Abstract
Dynamic treatment regimes are sequential decision rules that adapt throughout disease progression according to a patient's evolving characteristics. In many clinical applications, it is desirable that the format of the decision rules remains consistent over time. Unlike the estimation of dynamic treatment regimes in regular settings, where decision rules are formed without shared parameters, the derivation of the shared decision rules requires estimating shared parameters indexing the decision rules across different decision points. Estimation of such rules becomes more complicated when the clinical outcome of interest is a survival time subject to censoring. To address these challenges, we propose two novel methods: censored shared-Q-learning and censored shared-O-learning. Both methods incorporate clinical preferences into a qualitative rule, where the parameters indexing the decision rules are shared across different decision points and estimated simultaneously. We use simulation studies to demonstrate the superior performance of the proposed methods. The methods are further applied to the Framingham Heart Study to derive treatment rules for cardiovascular disease.
Collapse
Affiliation(s)
- Ying-Qi Zhao
- Public Health Sciences Division, Fred Hutchinson Cancer Research Center, Seattle, WA 98109, U.S.A
| | - Ruoqing Zhu
- Department of Statistics, University of Illinois Urbana-Champaign, Champaign, Illinois, 61820, U.S.A
| | - Guanhua Chen
- Department of Biostatistics and Medical Informatics, University of Wisconsin-Madison, Madison, Wisconsin, 53792, U.S.A
| | - Yingye Zheng
- Public Health Sciences Division, Fred Hutchinson Cancer Research Center, Seattle, WA 98109, U.S.A
| |
Collapse
|
18
|
Wu CS, Luedtke AR, Sadikova E, Tsai HJ, Liao SC, Liu CC, Gau SSF, VanderWeele TJ, Kessler RC. Development and Validation of a Machine Learning Individualized Treatment Rule in First-Episode Schizophrenia. JAMA Netw Open 2020; 3:e1921660. [PMID: 32083693 PMCID: PMC7043195 DOI: 10.1001/jamanetworkopen.2019.21660] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/07/2019] [Accepted: 12/23/2019] [Indexed: 12/31/2022] Open
Abstract
Importance Little guidance exists to date on how to select antipsychotic medications for patients with first-episode schizophrenia. Objective To develop a preliminary individualized treatment rule (ITR) for patients with first-episode schizophrenia. Design, Setting, and Participants This prognostic study obtained data from Taiwan's National Health Insurance Research Database on patients with prescribed antipsychotic medications, ambulatory claims, or discharge diagnoses of a schizophrenic disorder between January 1, 2005, and December 31, 2011. An ITR was developed by applying a targeted minimum loss-based ensemble machine learning method to predict treatment success from baseline clinical and demographic data in a 70% training sample. The model was validated in the remaining 30% of the sample. The probability of treatment success was estimated for each medication for each patient under the model. The analysis was conducted between July 16, 2018, and July 15, 2019. Exposures Fifteen different antipsychotic medications. Main Outcomes and Measures Treatment success was defined as not switching medication and not being hospitalized for 12 months. Results Among the 32 277 patients in the analysis, the mean (SD) age was 36.7 (14.3) years, and 15 752 (48.8%) were male. In the validation sample, the treatment success rate (SE) was 51.7% (1.0%) under the ITR and was 44.5% (0.5%) in the observed population (Z = 7.1; P < .001). The estimated treatment success if all patients were given a prescription for 1 medication was significantly lower for each of the 13 medications than under the ITR (Z = 4.2-16.8; all P < .001). Aripiprazole (3088 [31.9%]) and amisulpride (2920 [30.2%]) were the medications most often recommended by the ITR. Only 1054 patients (10.9%) received ITR-recommended medications. Observed treatment success, although lower than the success under the ITR, was nonetheless significantly higher than if medications had been randomized (44.5% [SE, 0.55%] vs 41.3% [SE, 0.4%]; Z = 6.9; P < .001), although only marginally higher than if medications had been randomized in their observed population proportions (44.5% [SE, 0.5%] vs 43.5% [SE, 0.4%]; Z = 2.2; P = .03]). Conclusions and Relevance These results suggest that an ITR may be associatded with an increase in the treatment success rate among patients with first-episode schizophrenia, but experimental evaluation is needed to confirm this possibility. If confirmed, model refinement that investigates biomarkers, clinical observations, and patient reports as additional predictors in iterative pragmatic trials would be needed before clinical implementation.
Collapse
Affiliation(s)
- Chi-Shin Wu
- Department of Psychiatry, National Taiwan University Hospital & College of Medicine, National Taiwan University, Taipei City, Taiwan
| | - Alex R. Luedtke
- Department of Statistics, University of Washington, Seattle
- Vaccine and Infectious Disease Division, Fred Hutchinson Cancer Research Center, Seattle, Washington
| | - Ekaterina Sadikova
- Department of Health Care Policy, Harvard Medical School, Boston, Massachusetts
- Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, Massachusetts
| | - Hui-Ju Tsai
- Institute of Population Health Sciences, National Health Research Institutes, Zhunan, Taiwan
| | - Shih-Cheng Liao
- Department of Psychiatry, National Taiwan University Hospital & College of Medicine, National Taiwan University, Taipei City, Taiwan
| | - Chen-Chung Liu
- Department of Psychiatry, National Taiwan University Hospital & College of Medicine, National Taiwan University, Taipei City, Taiwan
| | - Susan Shur-Fen Gau
- Department of Psychiatry, National Taiwan University Hospital & College of Medicine, National Taiwan University, Taipei City, Taiwan
| | - Tyler J. VanderWeele
- Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, Massachusetts
- Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, Massachusetts
| | - Ronald C. Kessler
- Department of Health Care Policy, Harvard Medical School, Boston, Massachusetts
| |
Collapse
|
19
|
Meng H, Zhao YQ, Fu H, Qiao X. Near-optimal Individualized Treatment Recommendations. JOURNAL OF MACHINE LEARNING RESEARCH : JMLR 2020; 21:183. [PMID: 34335111 PMCID: PMC8324003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Figures] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
The individualized treatment recommendation (ITR) is an important analytic framework for precision medicine. The goal of ITR is to assign the best treatments to patients based on their individual characteristics. From the machine learning perspective, the solution to the ITR problem can be formulated as a weighted classification problem to maximize the mean benefit from the recommended treatments given patients' characteristics. Several ITR methods have been proposed in both the binary setting and the multicategory setting. In practice, one may prefer a more flexible recommendation that includes multiple treatment options. This motivates us to develop methods to obtain a set of near-optimal individualized treatment recommendations alternative to each other, called alternative individualized treatment recommendations (A-ITR). We propose two methods to estimate the optimal A-ITR within the outcome weighted learning (OWL) framework. Simulation studies and a real data analysis for Type 2 diabetic patients with injectable antidiabetic treatments are conducted to show the usefulness of the proposed A-ITR framework. We also show the consistency of these methods and obtain an upper bound for the risk between the theoretically optimal recommendation and the estimated one. An R package aitr has been developed, found at https://github.com/menghaomiao/aitr.
Collapse
Affiliation(s)
- Haomiao Meng
- Department of Mathematical Sciences, Binghamton University, State University of New York, Binghamton, NY 13902, USA
| | - Ying-Qi Zhao
- Public Health Sciences Division, Fred Hutchinson Cancer Research Center, Seattle, WA 98109, USA
| | - Haoda Fu
- Eli Lilly and Company, Indianapolis, IN 46285, USA
| | - Xingye Qiao
- Department of Mathematical Sciences, Binghamton University, State University of New York, Binghamton, NY 13902, USA
| |
Collapse
|
20
|
Kessler RC, Bauer MS, Bishop TM, Demler OV, Dobscha SK, Gildea SM, Goulet JL, Karras E, Kreyenbuhl J, Landes SJ, Liu H, Luedtke AR, Mair P, McAuliffe WHB, Nock M, Petukhova M, Pigeon WR, Sampson NA, Smoller JW, Weinstock LM, Bossarte RM. Using Administrative Data to Predict Suicide After Psychiatric Hospitalization in the Veterans Health Administration System. Front Psychiatry 2020; 11:390. [PMID: 32435212 PMCID: PMC7219514 DOI: 10.3389/fpsyt.2020.00390] [Citation(s) in RCA: 25] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/13/2019] [Accepted: 04/17/2020] [Indexed: 12/11/2022] Open
Abstract
There is a very high suicide rate in the year after psychiatric hospital discharge. Intensive postdischarge case management programs can address this problem but are not cost-effective for all patients. This issue can be addressed by developing a risk model to predict which inpatients might need such a program. We developed such a model for the 391,018 short-term psychiatric hospital admissions of US veterans in Veterans Health Administration (VHA) hospitals 2010-2013. Records were linked with the National Death Index to determine suicide within 12 months of hospital discharge (n=771). The Super Learner ensemble machine learning method was used to predict these suicides for time horizon between 1 week and 12 months after discharge in a 70% training sample. Accuracy was validated in the remaining 30% holdout sample. Predictors included VHA administrative variables and small area geocode data linked to patient home addresses. The models had AUC=.79-.82 for time horizons between 1 week and 6 months and AUC=.74 for 12 months. An analysis of operating characteristics showed that 22.4%-32.2% of patients who died by suicide would have been reached if intensive case management was provided to the 5% of patients with highest predicted suicide risk. Positive predictive value (PPV) at this higher threshold ranged from 1.2% over 12 months to 3.8% per case manager year over 1 week. Focusing on the low end of the risk spectrum, the 40% of patients classified as having lowest risk account for 0%-9.7% of suicides across time horizons. Variable importance analysis shows that 51.1% of model performance is due to psychopathological risk factors accounted, 26.2% to social determinants of health, 14.8% to prior history of suicidal behaviors, and 6.6% to physical disorders. The paper closes with a discussion of next steps in refining the model and prospects for developing a parallel precision treatment model.
Collapse
Affiliation(s)
- Ronald C Kessler
- Deparment of Health Care Policy, Harvard Medical School, Boston, MA, United States
| | - Mark S Bauer
- Department of Psychiatry, Harvard Medical School, Boston, MA, United States.,Center for Healthcare Organization & Implementation Research, VA Boston Healthcare System, Boston, MA, United States
| | - Todd M Bishop
- Center of Excellence for Suicide Prevention, Canandaigua VA Medical Center, Canandaigua, NY, United States
| | - Olga V Demler
- Division of Preventive Medicine, Brigham and Women's Hospital, Boston, MA, United States.,Department of Medicine, Harvard Medical School, Boston, MA, United States
| | - Steven K Dobscha
- VA Center to Improve Veteran Involvement in Care, VA Portland Health Care System, Portland, OR, United States.,Department of Psychiatry, Oregon Health & Science University, Portland, OR, United States
| | - Sarah M Gildea
- Deparment of Health Care Policy, Harvard Medical School, Boston, MA, United States
| | - Joseph L Goulet
- Pain, Research, Informatics, Multimorbidities & Education Center, VA Connecticut Healthcare System, West Haven, CT, United States.,Department of Emergency Medicine, Yale School of Medicine, New Haven, CT, United States
| | - Elizabeth Karras
- Center of Excellence for Suicide Prevention, Canandaigua VA Medical Center, Canandaigua, NY, United States
| | - Julie Kreyenbuhl
- VA Capitol Healthcare Network (VISN 5), Mental Illness Research, Education, and Clinical Center (MIRECC), Baltimore, MD, United States.,Department of Psychiatry, Division of Psychiatric Services Research, University of Maryland School of Medicine, Baltimore, MD, United States
| | - Sara J Landes
- South Central Mental Illness Research Education Clinical Center (MIRECC), Central Arkansas Veterans Healthcare System, North Little Rock, AR, United States.,Department of Psychiatry, University of Arkansas for Medical Sciences, Little Rock, AR, United States
| | - Howard Liu
- Deparment of Health Care Policy, Harvard Medical School, Boston, MA, United States.,Center of Excellence for Suicide Prevention, Canandaigua VA Medical Center, Canandaigua, NY, United States
| | - Alex R Luedtke
- Department of Statistics, University of Washington, Seattle, WA, United States.,Vaccine and Infectious Disease Division, Fred Hutchinson Cancer Research Center, Seattle, WA, United States
| | - Patrick Mair
- Department of Psychology, Harvard University, Cambridge, MA, United States
| | | | - Matthew Nock
- Department of Psychology, Harvard University, Cambridge, MA, United States
| | - Maria Petukhova
- Deparment of Health Care Policy, Harvard Medical School, Boston, MA, United States
| | - Wilfred R Pigeon
- Center of Excellence for Suicide Prevention, Canandaigua VA Medical Center, Canandaigua, NY, United States.,Department of Psychiatry, University of Rochester Medical Center, Rochester, NY, United States
| | - Nancy A Sampson
- Deparment of Health Care Policy, Harvard Medical School, Boston, MA, United States
| | - Jordan W Smoller
- Department of Psychiatry, Massachusetts General Hospital, Boston, MA, United States
| | - Lauren M Weinstock
- Department of Psychiatry & Human Behavior, Alpert Medical School of Brown University, Providence, RI, United States
| | - Robert M Bossarte
- Center of Excellence for Suicide Prevention, Canandaigua VA Medical Center, Canandaigua, NY, United States.,West Virginia University Injury Control Research Center and Department of Behavioral Medicine and Psychiatry, West Virginia University School of Medicine, Morgantown, WV, United States
| |
Collapse
|
21
|
Kessler RC, Bossarte RM, Luedtke A, Zaslavsky AM, Zubizarreta JR. Machine learning methods for developing precision treatment rules with observational data. Behav Res Ther 2019; 120:103412. [PMID: 31233922 DOI: 10.1016/j.brat.2019.103412] [Citation(s) in RCA: 32] [Impact Index Per Article: 6.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/13/2019] [Revised: 05/15/2019] [Accepted: 05/26/2019] [Indexed: 12/28/2022]
Abstract
Clinical trials have identified a variety of predictor variables for use in precision treatment protocols, ranging from clinical biomarkers and symptom profiles to self-report measures of various sorts. Although such variables are informative collectively, none has proven sufficiently powerful to guide optimal treatment selection individually. This has prompted growing interest in the development of composite precision treatment rules (PTRs) that are constructed by combining information across a range of predictors. But this work has been hampered by the generally small samples in randomized clinical trials and the use of suboptimal analysis methods to analyze the resulting data. In this paper, we propose to address the sample size problem by: working with large observational electronic medical record databases rather than controlled clinical trials to develop preliminary PTRs; validating these preliminary PTRs in subsequent pragmatic trials; and using ensemble machine learning methods rather than individual algorithms to carry out statistical analyses to develop the PTRs. The major challenges in this proposed approach are that treatment are not randomly assigned in observational databases and that these databases often lack measures of key prescriptive predictors and mental disorder treatment outcomes. We proposed a tiered case-cohort design approach that uses innovative methods for measuring and balancing baseline covariates and estimating PTRs to address these challenges.
Collapse
Affiliation(s)
- Ronald C Kessler
- Department of Health Care Policy, Harvard Medical School, Boston, MA, USA.
| | - Robert M Bossarte
- West Virginia University Injury Control Research Center, Morgantown, WV, USA; Department of Behavioral Medicine and Psychiatry, West Virginia University, Morgantown, WV, USA; VISN 2 Center of Excellence for Suicide Prevention, Canandaigua VA Medical Center, Canandaigua, NY, USA
| | - Alex Luedtke
- Department of Statistics, University of Washington, Seattle, WA, USA; Vaccine and Infectious Disease Division, Fred Hutchinson Cancer Research Center, Seattle, WA, USA
| | - Alan M Zaslavsky
- Department of Health Care Policy, Harvard Medical School, Boston, MA, USA
| | - Jose R Zubizarreta
- Department of Health Care Policy, Harvard Medical School, Boston, MA, USA; Department of Statistics, Faculty of Arts and Sciences, Harvard University, Cambridge, MA, USA
| |
Collapse
|
22
|
Huang X, Goldberg Y, Xu J. Multicategory individualized treatment regime using outcome weighted learning. Biometrics 2019; 75:1216-1227. [PMID: 31095722 DOI: 10.1111/biom.13084] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2018] [Accepted: 05/09/2019] [Indexed: 12/01/2022]
Abstract
Individualized treatment regimes (ITRs) aim to recommend treatments based on patient-specific characteristics in order to maximize the expected clinical outcome. Outcome weighted learning approaches have been proposed for this optimization problem with primary focus on the binary treatment case. Many require assumptions of the outcome value or the randomization mechanism. In this paper, we propose a general framework for multicategory ITRs using generic surrogate risk. The proposed method accommodates the situations when the outcome takes negative value and/or when the propensity score is unknown. Theoretical results about Fisher consistency, excess risk, and risk consistency are established. In practice, we recommend using differentiable convex loss for computational optimization. We demonstrate the superiority of the proposed method under multinomial deviance risk to some existing methods by simulation and application on data from a clinical trial.
Collapse
Affiliation(s)
- Xinyang Huang
- Key Laboratory of Advanced Theory and Application in Statistics and Data Science-MOE, and School of Statistics, East China Normal University, Shanghai, China
| | - Yair Goldberg
- The Faculty of Industrial Engineering and Management, Technion-Israel Institute of Technology, Haifa, Israel
| | - Jin Xu
- Key Laboratory of Advanced Theory and Application in Statistics and Data Science-MOE, and School of Statistics, East China Normal University, Shanghai, China
| |
Collapse
|
23
|
Mi X, Zou F, Zhu R. Bagging and deep learning in optimal individualized treatment rules. Biometrics 2019; 75:674-684. [PMID: 30365175 DOI: 10.1111/biom.12990] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2017] [Accepted: 10/09/2018] [Indexed: 11/30/2022]
Abstract
An ENsemble Deep Learning Optimal Treatment (EndLot) approach is proposed for personalized medicine problems. The statistical framework of the proposed method is based on the outcome weighted learning (OWL) framework which transforms the optimal decision rule problem into a weighted classification problem. We further employ an ensemble of deep neural networks (DNNs) to learn the optimal decision rule. Utilizing the flexibility of DNNs and the stability of bootstrap aggregation, the proposed method achieves a considerable improvement over existing methods. An R package "ITRlearn" is developed to implement the proposed method. Numerical performance is demonstrated via simulation studies and a real data analysis of the Cancer Cell Line Encyclopedia data.
Collapse
Affiliation(s)
- Xinlei Mi
- Department of Biostatistics, University of Florida, Gainesville, Florida
| | - Fei Zou
- Department of Biostatistics, University of Florida, Gainesville, Florida
| | - Ruoqing Zhu
- Department of Statistics, University of Illinois at Urbana-Champaign, Champaign, Illinois
| |
Collapse
|