Estimating Individualized Treatment Rules Using Outcome Weighted Learning

Transfer learning of individualized treatment rules from experimental to real-world data

Individualized treatment effect lies at the heart of precision medicine. Interpretable individualized treatment rules (ITRs) are desirable for clinicians or policymakers due to their intuitive appeal and transparency. The gold-standard approach to estimating the ITRs is randomized experiments, where subjects are randomized to different treatment groups and the confounding bias is minimized to the extent possible. However, experimental studies are limited in external validity because of their selection restrictions, and therefore the underlying study population is not representative of the target real-world population. Conventional learning methods of optimal interpretable ITRs for a target population based only on experimental data are biased. On the other hand, real-world data (RWD) are becoming popular and provide a representative sample of the target population. To learn the generalizable optimal interpretable ITRs, we propose an integrative transfer learning method based on weighting schemes to calibrate the covariate distribution of the experiment to that of the RWD. Theoretically, we establish the risk consistency for the proposed ITR estimator. Empirically, we evaluate the finite-sample performance of the transfer learner through simulations and apply it to a real data application of a job training program.

Contrast weighted learning for robust optimal treatment rule estimation

Personalized medicine aims to tailor medical decisions based on patient-specific characteristics. Advances in data capturing techniques such as electronic health records dramatically increase the availability of comprehensive patient profiles, promoting the rapid development of optimal treatment rule (OTR) estimation methods. An archetypal OTR estimation approach is the outcome weighted learning, where OTR is determined under a weighted classification framework with clinical outcomes as the weights. Although outcome weighted learning has been extensively studied and extended, existing methods are susceptible to irregularities of outcome distributions such as outliers and heavy tails. Methods that involve modeling of the outcome are also sensitive to model misspecification. We propose a contrast weighted learning (CWL) framework that exploits the flexibility and robustness of contrast functions to enable robust OTR estimation for a wide range of clinical outcomes. The novel value function in CWL only depends on the pairwise contrast of clinical outcomes between patients irrespective of their distributional features and supports. The Fisher consistency and convergence rate of the estimated decision rule via CWL are established. We illustrate the superiority of the proposed method under finite samples using comprehensive simulation studies with ill-distributed continuous outcomes and ordinal outcomes. We apply the CWL method to two datasets from clinical trials on idiopathic pulmonary fibrosis and COVID-19 to demonstrate its real-world application.

Meta-analysis of individualized treatment rules via sign-coherency

Medical treatments tailored to a patient's baseline characteristics hold the potential of improving patient outcomes while reducing negative side effects. Learning individualized treatment rules (ITRs) often requires aggregation of multiple datasets(sites); however, current ITR methodology does not take between-site heterogeneity into account, which can hurt model generalizability when deploying back to each site. To address this problem, we develop a method for individual-level meta-analysis of ITRs, which jointly learns site-specific ITRs while borrowing information about feature sign-coherency via a scientifically-motivated directionality principle. We also develop an adaptive procedure for model tuning, using information criteria tailored to the ITR learning problem. We study the proposed methods through numerical experiments to understand their performance under different levels of between-site heterogeneity and apply the methodology to estimate ITRs in a large multi-center database of electronic health records. This work extends several popular methodologies for estimating ITRs (A-learning, weighted learning) to the multiple-sites setting.

Predictive signature development based on maximizing the area between receiver operating characteristic curves

Development of marker signatures to predict treatment benefits for a new therapeutic is an important scientific component in advancing the drug discovery and is an important first step toward the goal of precision medicine. In this article, we focus on developing an algorithm to search for optimal linear combination of markers that maximizes the area between two receiver operating characteristic curves of the new therapeutic and the control groups without assuming any parametric model. We further generalize the proposed algorithm for predictive signature development to maximize the difference of Harrel's C-index of the new therapeutic and the control groups when the outcome of interest is time-to-event. The performance of this proposed method is evaluated and compared to existing methods via simulations and real clinical trial data.

An efficient approach for optimizing the cost-effective individualized treatment rule using conditional random forest

Evidence from observational studies has become increasingly important for supporting healthcare policy making via cost-effectiveness analyses. Similar as in comparative effectiveness studies, health economic evaluations that consider subject-level heterogeneity produce individualized treatment rules that are often more cost-effective than one-size-fits-all treatment. Thus, it is of great interest to develop statistical tools for learning such a cost-effective individualized treatment rule under the causal inference framework that allows proper handling of potential confounding and can be applied to both trials and observational studies. In this paper, we use the concept of net-monetary-benefit to assess the trade-off between health benefits and related costs. We estimate cost-effective individualized treatment rule as a function of patients' characteristics that, when implemented, optimizes the allocation of limited healthcare resources by maximizing health gains while minimizing treatment-related costs. We employ the conditional random forest approach and identify the optimal cost-effective individualized treatment rule using net-monetary-benefit-based classification algorithms, where two partitioned estimators are proposed for the subject-specific weights to effectively incorporate information from censored individuals. We conduct simulation studies to evaluate the performance of our proposals. We apply our top-performing algorithm to the NIH-funded Systolic Blood Pressure Intervention Trial to illustrate the cost-effectiveness gains of assigning customized intensive blood pressure therapy.

Semiparametric single-index models for optimal treatment regimens with censored outcomes

There is a growing interest in precision medicine, where a potentially censored survival time is often the most important outcome of interest. To discover optimal treatment regimens for such an outcome, we propose a semiparametric proportional hazards model by incorporating the interaction between treatment and a single index of covariates through an unknown monotone link function. This model is flexible enough to allow non-linear treatment-covariate interactions and yet provides a clinically interpretable linear rule for treatment decision. We propose a sieve maximum likelihood estimation approach, under which the baseline hazard function is estimated nonparametrically and the unknown link function is estimated via monotone quadratic B-splines. We show that the resulting estimators are consistent and asymptotically normal with a covariance matrix that attains the semiparametric efficiency bound. The optimal treatment rule follows naturally as a linear combination of the maximum likelihood estimators of the model parameters. Through extensive simulation studies and an application to an AIDS clinical trial, we demonstrate that the treatment rule derived from the single-index model outperforms the treatment rule under the standard Cox proportional hazards model.

A Comparison of Methods for Treatment Assignment with an Application to Playlist Generation

This study presents a systematic comparison of methods for individual treatment assignment. We group the various methods proposed in the literature into three general classes of algorithms (or metalearners): learning models to predict outcomes (the O-learner), learning models to predict causal effects (the E-learner), and learning models to predict optimal treatment assignments (the A-learner). We discuss how the metalearners differ in their level of generality and their objective function, which has critical implications for modeling and decision making. Notably, we demonstrate that optimizing for the prediction of outcomes or causal effects is not the same as optimizing for treatment assignments, suggesting that, in general, the A-learner should lead to better treatment assignments than the other metalearners. We then compare the metalearners in the context of choosing, for each user, the best algorithm for playlist generation in order to optimize engagement. This is the first comparison of the three different metalearners on a real-world application at scale (based on more than half a billion treatment assignments). In addition to supporting our analytical findings, the results show how large A/B tests can provide substantial value for learning treatment-assignment policies, rather than simply choosing the variant that performs best on average.

Quantiles based personalized treatment selection for multivariate outcomes and multiple treatments

In this work, we propose a method for individualized treatment selection when there are correlated multiple responses for the K treatment ( K ≥ 2 ) scenario. Here we use ranks of quantiles of outcome variables for each treatment conditional on patient-specific scores constructed from collected covariate measurements. Our method covers any number of treatments and outcome variables using any number of quantiles and it can be applied for a broad set of models. We propose a rank aggregation technique for combining several lists of ranks where both these lists and elements within each list can be correlated. The method has the flexibility to incorporate patient and clinician preferences into the optimal treatment decision on an individual case basis. A simulation study demonstrates the performance of the proposed method in finite samples. We also present illustrations using two different datasets from diabetes and HIV-1 clinical trials to show the applicability of the proposed procedure for real data.

Deep reinforcement learning for personalized treatment recommendation

In precision medicine, the ultimate goal is to recommend the most effective treatment to an individual patient based on patient-specific molecular and clinical profiles, possibly high-dimensional. To advance cancer treatment, large-scale screenings of cancer cell lines against chemical compounds have been performed to help better understand the relationship between genomic features and drug response; existing machine learning approaches use exclusively supervised learning, including penalized regression and recommender systems. However, it would be more efficient to apply reinforcement learning to sequentially learn as data accrue, including selecting the most promising therapy for a patient given individual molecular and clinical features and then collecting and learning from the corresponding data. In this article, we propose a novel personalized ranking system called Proximal Policy Optimization Ranking (PPORank), which ranks the drugs based on their predicted effects per cell line (or patient) in the framework of deep reinforcement learning (DRL). Modeled as a Markov decision process, the proposed method learns to recommend the most suitable drugs sequentially and continuously over time. As a proof-of-concept, we conduct experiments on two large-scale cancer cell line data sets in addition to simulated data. The results demonstrate that the proposed DRL-based PPORank outperforms the state-of-the-art competitors based on supervised learning. Taken together, we conclude that novel methods in the framework of DRL have great potential for precision medicine and should be further studied.

Selecting predictive biomarkers from genomic data

Recently there have been tremendous efforts to develop statistical procedures which allow to determine subgroups of patients for which certain treatments are effective. This article focuses on the selection of prognostic and predictive genetic biomarkers based on a relatively large number of candidate Single Nucleotide Polymorphisms (SNPs). We consider models which include prognostic markers as main effects and predictive markers as interaction effects with treatment. We compare different high-dimensional selection approaches including adaptive lasso, a Bayesian adaptive version of the Sorted L-One Penalized Estimator (SLOBE) and a modified version of the Bayesian Information Criterion (mBIC2). These are compared with classical multiple testing procedures for individual markers. Having identified predictive markers we consider several different approaches how to specify subgroups susceptible to treatment. Our main conclusion is that selection based on mBIC2 and SLOBE has similar predictive performance as the adaptive lasso while including substantially fewer biomarkers.

The optimal dynamic treatment rule superlearner: considerations, performance, and application to criminal justice interventions

The optimal dynamic treatment rule (ODTR) framework offers an approach for understanding which kinds of patients respond best to specific treatments - in other words, treatment effect heterogeneity. Recently, there has been a proliferation of methods for estimating the ODTR. One such method is an extension of the SuperLearner algorithm - an ensemble method to optimally combine candidate algorithms extensively used in prediction problems - to ODTRs. Following the ``causal roadmap," we causally and statistically define the ODTR and provide an introduction to estimating it using the ODTR SuperLearner. Additionally, we highlight practical choices when implementing the algorithm, including choice of candidate algorithms, metalearners to combine the candidates, and risk functions to select the best combination of algorithms. Using simulations, we illustrate how estimating the ODTR using this SuperLearner approach can uncover treatment effect heterogeneity more effectively than traditional approaches based on fitting a parametric regression of the outcome on the treatment, covariates and treatment-covariate interactions. We investigate the implications of choices in implementing an ODTR SuperLearner at various sample sizes. Our results show the advantages of: (1) including a combination of both flexible machine learning algorithms and simple parametric estimators in the library of candidate algorithms; (2) using an ensemble metalearner to combine candidates rather than selecting only the best-performing candidate; (3) using the mean outcome under the rule as a risk function. Finally, we apply the ODTR SuperLearner to the ``Interventions" study, an ongoing randomized controlled trial, to identify which justice-involved adults with mental illness benefit most from cognitive behavioral therapy to reduce criminal re-offending.

A first step towards best practice recommendations for the design and statistical analyses of pragmatic clinical trials: a modified Delphi approach

AIM

Pragmatic clinical trials (PCTs) are randomised trials implemented through routine clinical practice, where design parameters of traditional randomised controlled trials are modified to increase generalizability. However, this may introduce statistical challenges. We aimed to identify these challenges and discuss possible solutions leading to best practice recommendations for the design and analysis of PCTs.

METHODS

A modified Delphi method was used to reach consensus among a panel of 11 experts in clinical trials and statistics. Statistical issues were identified in a focused literature review and aggregated with insights and possible solutions from expert collected through a series of survey iterations. Issues were ranked according to their importance.

RESULTS

27 articles were included and combined with experts' insight to generate a list of issues categorized into: participants; recruiting sites; randomisation, blinding and intervention; outcome (selection and measurement); and data analysis. Consensus was reached about the most important issues: risk of participants' attrition; heterogeneity of "usual care" across sites; absence of blinding; use of a subjective endpoint; and data analysis aligned with the trial estimand. Potential issues should be anticipated and preferably be addressed in the trial protocol. The experts provided solutions regarding data collection and data analysis, which were considered of equal importance.

DISCUSSION

A set of important statistical issues in PCTs was identified and approaches were suggested to anticipate and/or minimize these through data analysis. Any impact of choosing a pragmatic design feature should be gauged in the light of the trial estimand.

Precision radiotherapy via information integration of expert human knowledge and AI recommendation to optimize clinical decision making

In the precision medicine era, there is a growing need for precision radiotherapy where the planned radiation dose needs to be optimally determined by considering a myriad of patient-specific information in order to ensure treatment efficacy. Existing artificial-intelligence (AI) methods can recommend radiation dose prescriptions within the scope of this available information. However, treating physicians may not fully entrust the AI's recommended prescriptions due to known limitations or at instances when the AI recommendation may go beyond physicians' current knowledge. This paper lays out a systematic method to integrate expert human knowledge with AI recommendations for optimizing clinical decision making. Towards this goal, Gaussian process (GP) models are integrated with deep neural networks (DNNs) to quantify the uncertainty of the treatment outcomes given by physicians and AI recommendations, respectively, which are further used as a guideline to educate clinical physicians and improve AI models performance. The proposed method is demonstrated in a comprehensive dataset where patient-specific information and treatment outcomes are prospectively collected during radiotherapy of 67 non-small cell lung cancer (NSCLC) patients and are retrospectively analyzed.

Model-free screening for variables with treatment interaction

Precision medicine is a medical paradigm that focuses on making effective treatment decision based on individual patient characteristics. When there are a large amount of patient information, such as patient’s genetic information, medical records and clinical measurements, available, it is of interest to select the covariates which have interactions with the treatment, for example, in determining the individualized treatment regime where only a subset of covariates with treatment interactions involves in decision making. We propose a marginal feature ranking and screening procedure for measuring interactions between the treatment and covariates. The method does not require imposing a specific model structure on the regression model and is applicable in a high dimensional setting. Theoretical properties in terms of consistency in ranking and selection are established. We demonstrate the finite sample performance of the proposed method by simulation and illustrate the applications with two real data examples from clinical trials.

A Machine-Learning Approach for Estimating Subgroup- and Individual-Level Treatment Effects: An Illustration Using the 65 Trial

Personalizing treatment recommendations or guidelines requires evidence about the heterogeneity of treatment effects (HTE). Machine-learning (ML) approaches can explore HTE by considering many covariates, including complex interactions between them. Causal ML approaches can avoid overfitting, which arises when the same dataset is used to select covariate by treatment interaction terms as to make inferences and reduce reliance on the correct specification of fixed parametric models. We investigate causal forests (CF), a ML method based on modified decision trees that can estimate subgroup- and individual-level treatment effects, without requiring correct prespecification of the effect model. We consider CF alongside parametric approaches for estimating HTE, within the 65 Trial, which evaluates the effect of a permissive hypotension strategy versus usual care on 90-d mortality for critically ill patients aged 65 y or older with vasodilatory hypotension. Here, the CF approach provides similar estimates of treatment effectiveness for prespecified and post hoc subgroups to the parametric approach, and the results of a test for overall HTE show weak evidence of heterogeneity. The CF estimates of individual-level treatment effects, the expected effects of treatment for individuals in subpopulations defined by their covariates, suggest that the permissive hypotension strategy is expected to reduce 90-d mortality for 98.7% of patients but with 95% confidence intervals that include zero for 71.6% of patients. A ML approach is then used to assess the patient characteristics associated with these individual-level effects, and to help target future research that can identify those patient subgroups for whom the intervention is most effective.

Multiple domain and multiple kernel outcome-weighted learning for estimating individualized treatment regimes

Individualized treatment rules (ITRs) recommend treatments that are tailored specifically according to each patient's own characteristics. It can be challenging to estimate optimal ITRs when there are many features, especially when these features have arisen from multiple data domains (e.g., demographics, clinical measurements, neuroimaging modalities). Considering data from complementary domains and using multiple similarity measures to capture the potential complex relationship between features and treatment can potentially improve the accuracy of assigning treatments. Outcome weighted learning (OWL) methods that are based on support vector machines using a predetermined single kernel function have previously been developed to estimate optimal ITRs. In this paper, we propose an approach to estimate optimal ITRs by exploiting multiple kernel functions to describe the similarity of features between subjects both within and across data domains within the OWL framework, as opposed to preselecting a single kernel function to be used for all features for all domains. Our method takes into account the heterogeneity of each data domain and combines multiple data domains optimally. Our learning process estimates optimal ITRs and also identifies the data domains that are most important for determining ITRs. This approach can thus be used to prioritize the collection of data from multiple domains, potentially reducing cost without sacrificing accuracy. The comparative advantage of our method is demonstrated by simulation studies and by an application to a randomized clinical trial for major depressive disorder that collected features from multiple data domains. Supplemental materials for this article are available online.

Self-matched learning to construct treatment decision rules from electronic health records

Electronic health records (EHRs) collected from large-scale health systems provide rich subject-specific information on a broad patient population at a lower cost compared to randomized controlled trials. Thus, EHRs may serve as a complementary resource to provide real-world data to construct individualized treatment rules (ITRs) and achieve precision medicine. However, in the absence of randomization, inferring treatment rules from EHR data may suffer from unmeasured confounding. In this article, we propose a self-matched learning method inspired by the self-controlled case series (SCCS) design to mitigate this challenge. We alleviate unmeasured time-invariant confounding between patients by matching different periods of treatments within the same patient (self-controlled matching) to infer the optimal ITRs. The proposed method constructs a within-subject matched value function for optimizing ITRs and bears similarity to the SCCS design. We examine assumptions that ensure Fisher consistency, and show that our method requires weaker assumptions on unmeasured confounding than alternative methods. Through extensive simulation studies, we demonstrate that self-matched learning has comparable performance to other existing methods when there are no unmeasured confounders, but performs markedly better when unobserved time-invariant confounders are present, which is often the case for EHRs. Sensitivity analyses show that the proposed method is robust under different scenarios. Finally, we apply self-matched learning to estimate the optimal ITRs from type 2 diabetes patient EHRs, which shows our estimated decision rules lead to greater advantages in reducing patients' diabetes-related complications.

A sparse additive model for treatment effect-modifier selection

Sparse additive modeling is a class of effective methods for performing high-dimensional nonparametric regression. This article develops a sparse additive model focused on estimation of treatment effect modification with simultaneous treatment effect-modifier selection. We propose a version of the sparse additive model uniquely constrained to estimate the interaction effects between treatment and pretreatment covariates, while leaving the main effects of the pretreatment covariates unspecified. The proposed regression model can effectively identify treatment effect-modifiers that exhibit possibly nonlinear interactions with the treatment variable that are relevant for making optimal treatment decisions. A set of simulation experiments and an application to a dataset from a randomized clinical trial are presented to demonstrate the method.

On Robustness of Individualized Decision Rules

With the emergence of precision medicine, estimating optimal individualized decision rules (IDRs) has attracted tremendous attention in many scientific areas. Most existing literature has focused on finding optimal IDRs that can maximize the expected outcome for each individual. Motivated by complex individualized decision making procedures and the popular conditional value at risk (CVaR) measure, we propose a new robust criterion to estimate optimal IDRs in order to control the average lower tail of the individuals' outcomes. In addition to improving the individualized expected outcome, our proposed criterion takes risks into consideration, and thus the resulting IDRs can prevent adverse events. The optimal IDR under our criterion can be interpreted as the decision rule that maximizes the "worst-case" scenario of the individualized outcome when the underlying distribution is perturbed within a constrained set. An efficient non-convex optimization algorithm is proposed with convergence guarantees. We investigate theoretical properties for our estimated optimal IDRs under the proposed criterion such as consistency and finite sample error bounds. Simulation studies and a real data application are used to further demonstrate the robust performance of our methods. Several extensions of the proposed method are also discussed.

Utility based approach in individualized optimal dose selection using machine learning methods

The goal in personalized medicine is to individualize treatment using patient characteristics and improve health outcomes. Selection of optimal dose must balance the effect of dose on both treatment efficacy and toxicity outcomes. We consider a setting with one binary efficacy and one binary toxicity outcome. The goal is to find the optimal dose for each patient using clinical features and biomarkers from available dataset. We propose to use flexible machine learning methods such as random forest and Gaussian process models to build models for efficacy and toxicity depending on dose and biomarkers. A copula is used to model the joint distribution of the two outcomes and the estimates are constrained to have non‐decreasing dose‐efficacy and dose‐toxicity relationships. Numerical utilities are elicited from clinicians for each potential bivariate outcome. For each patient, the optimal dose is chosen to maximize the posterior mean of the utility function. We also propose alternative approaches to optimal dose selection by adding additional toxicity based constraints and an approach taking into account the uncertainty in the estimation of the utility function. The proposed methods are evaluated in a simulation study to compare expected utility outcomes under various estimated optimal dose rules. Gaussian process models tended to have better performance than random forest. Enforcing monotonicity during modeling provided small benefits. Whether and how, correlation between efficacy and toxicity, was modeled, had little effect on performance. The proposed methods are illustrated with a study of patients with liver cancer treated with stereotactic body radiation therapy.

Novel Use of Patient-Specific Covariates From Oncology Studies in the Era of Biomedical Data Science: A Review of Latest Methodologies

In this article, we review different applications of how to incorporate individual patient variables into clinical research within oncology. These methodologies range from the more traditional use of baseline covariates from randomized clinical trials, as well as observational studies, to using covariates to generalize the results of randomized clinical trials to other populations. Individual patient variables also allow for the consideration of heterogeneity in treatment effects and individualized treatment rules. We primarily consider two treatment groups and mostly focus on time-to-event outcomes where such methodologies have been well established and widely applied. We also discuss more conceptually newer statistical research that has not been widely applied in clinical oncology, but is likely to make an impact in future oncology research. With the increasing amount of biomedical data available for analysis, it is inevitable that more methods are developed to make best use of information, to advance oncology research.

Estimating the optimal individualized treatment rule from a cost-effectiveness perspective

Optimal individualized treatment rules (ITRs) provide customized treatment recommendations based on subject characteristics to maximize clinical benefit in accordance with the objectives in precision medicine. As a result, there is growing interest in developing statistical tools for estimating optimal ITRs in evidence-based research. In health economic perspectives, policy makers consider the tradeoff between health gains and incremental costs of interventions to set priorities and allocate resources. However, most work on ITRs has focused on maximizing the effectiveness of treatment without considering costs. In this paper, we jointly consider the impact of effectiveness and cost on treatment decisions and define ITRs under a composite-outcome setting, so that we identify the most cost-effective ITR that accounts for individual-level heterogeneity through direct optimization. In particular, we propose a decision-tree-based statistical learning algorithm that uses a net-monetary-benefit-based reward to provide nonparametric estimations of the optimal ITR. We provide several approaches to estimating the reward underlying the ITR as a function of subject characteristics. We present the strengths and weaknesses of each approach and provide practical guidelines by comparing their performance in simulation studies. We illustrate the top-performing approach from our simulations by evaluating the projected 15-year personalized cost-effectiveness of the intensive blood pressure control of the Systolic Blood Pressure Intervention Trial (SPRINT) study.

Ranking tailoring variables for constructing individualized treatment rules: an application to schizophrenia

As with many chronic conditions, matching patients with schizophrenia to the best treatment options is difficult. Selecting antipsychotic medication is especially challenging because many of the medications can have burdensome side effects. Adjusting or tailoring medications based on patients' characteristics could improve symptoms. However, it is often not known which patient characteristics are most helpful for informing treatment selection. In this paper, we address the challenge of identifying and ranking important variables for tailoring treatment decisions. We consider a value-search approach implemented through dynamic marginal structural models to estimate an optimal individualized treatment rule. We apply our methodology to the Clinical Antipsychotics Trial of Intervention and Effectiveness (CATIE) study for schizophrenia, to evaluate if some tailoring variables have greater potential than others for selecting treatments for patients with schizophrenia (Stroup et al., 2003).

The Opioid Analgesic Reduction Study (OARS)-a comparison of opioid vs. non-opioid combination analgesics for management of post-surgical pain: a double-blind randomized clinical trial

Background

Everyday people die unnecessarily from opioid overdose-related addiction. Dentists are among the leading prescribers of opioid analgesics. Opioid-seeking behaviors have been linked to receipt of initial opioid prescriptions following the common dental procedure of third molar extraction. With each opioid prescription, a patient’s risk for opioid misuse or abuse increases. With an estimated 56 million tablets of 5 mg hydrocodone annually prescribed after third molar extractions in the USA, 3.5 million young adults may be unnecessarily exposed to opioids by dentists who are inadvertently increasing their patient’s risk for addiction.

Methods

A double-blind, stratified randomized, multi-center clinical trial has been designed to evaluate whether a combination of over-the-counter non-opioid-containing analgesics is not inferior to the most prescribed opioid analgesic. The impacted 3rd molar extraction model is being used due to the predictable severity of the post-operative pain and generalizability of results. Within each site/clinic and gender type (male/female), patients are randomized to receive either OPIOID (hydrocodone/acetaminophen 5/300 mg) or NON-OPIOID (ibuprofen/acetaminophen 400/500 mg). Outcome data include pain levels, adverse events, overall patient satisfaction, ability to sleep, and ability to perform daily functions. To develop clinical guidelines and a clinical decision-making tool, pain management, extraction difficulty, and the number of tablets taken are being collected, enabling an experimental decision-making tool to be developed.

Discussion

The proposed methods address the shortcomings of other analgesic studies. Although prior studies have tested short-term effects of single doses of pain medications, patients and their dentists are interested in managing pain for the entire post-operative period, not just the first 12 h. After surgery, patients expect to be able to perform normal daily functions without feeling nauseous or dizzy and they desire a restful sleep at night. Parents of young people are concerned with the risks of opioid use and misuse, related either to treatments received or to subsequent use of leftover pills. Upon successful completion of this clinical trial, dentists, patients, and their families will be better able to make informed decisions regarding post-operative pain management.

Trial registration

ClinicalTrials.govNCT04452344. Registered on June 20, 2020

Learning dynamic treatment strategies for coronary heart diseases by artificial intelligence: real-world data-driven study

BACKGROUND

Coronary heart disease (CHD) has become the leading cause of death and one of the most serious epidemic diseases worldwide. CHD is characterized by urgency, danger and severity, and dynamic treatment strategies for CHD patients are needed. We aimed to build and validate an AI model for dynamic treatment recommendations for CHD patients with the goal of improving patient outcomes and learning best practices from clinicians to help clinical decision support for treating CHD patients.

METHODS

We formed the treatment strategy as a sequential decision problem, and applied an AI supervised reinforcement learning-long short-term memory (SRL-LSTM) framework that combined supervised learning (SL) and reinforcement learning (RL) with an LSTM network to track patients' states to learn a recommendation model that took a patient's diagnosis and evolving health status as input and provided a treatment recommendation in the form of whether to take specific drugs. The experiments were conducted by leveraging a real-world intensive care unit (ICU) database with 13,762 admitted patients diagnosed with CHD. We compared the performance of the applied SRL-LSTM model and several state-of-the-art SL and RL models in reducing the estimated in-hospital mortality and the Jaccard similarity with clinicians' decisions. We used a random forest algorithm to calculate the feature importance of both the clinician policy and the AI policy to illustrate the interpretability of the AI model.

RESULTS

Our experimental study demonstrated that the AI model could help reduce the estimated in-hospital mortality through its RL function and learn the best practice from clinicians through its SL function. The similarity between the clinician policy and the AI policy regarding the surviving patients was high, while for the expired patients, it was much lower. The dynamic treatment strategies made by the AI model were clinically interpretable and relied on sensible clinical features extracted according to monitoring indexes and risk factors for CHD patients.

CONCLUSIONS

We proposed a pipeline for constructing an AI model to learn dynamic treatment strategies for CHD patients that could improve patient outcomes and mimic the best practices of clinicians. And a lot of further studies and efforts are needed to make it practical.

Preserving data privacy when using multi-site data to estimate individualized treatment rules

Precision medicine is a rapidly expanding area of health research wherein patient level information is used to inform treatment decisions. A statistical framework helps to formalize the individualization of treatment decisions that characterize personalized management plans. Numerous methods have been proposed to estimate individualized treatment rules that optimize expected patient outcomes, many of which have desirable properties such as robustness to model misspecification. However, while individual data are essential in this context, there may be concerns about data confidentiality, particularly in multi-center studies where data are shared externally. To address this issue, we compared two approaches to privacy preservation: (i) data pooling, which is a covariate microaggregation technique and (ii) distributed regression. These approaches were combined with the doubly robust yet user-friendly method of dynamic weighted ordinary least squares to estimate individualized treatment rules. In simulations, we extensively evaluated the performance of the methods in estimating the parameters of the decision rule under different assumptions. The results demonstrate that double robustness is not maintained in data pooling setting and that this can result in bias, whereas the distributed regression provides good performance. We illustrate the methods via an analysis of optimal Warfarin dosing using data from the International Warfarin Consortium.

Personalized treatment selection using observational data

Estimating the optimal treatment regime based on individual patient characteristics has been a topic of discussion in many forums. Advanced computational power has added momentum to this discussion over the last two decades and practitioners have been advocating the use of new methods in determining the best treatment. Treatments that are geared toward the 'best' outcome for a patient based on his/her genetic markers and characteristics are of high importance. In this article, we develop an approach to predict the optimal personalized treatment based on observational data. We have used inverse probability of treatment weighted machine learning methods to obtain score functions to predict the optimal treatment. Extensive simulation studies showed that our proposed method has desirable performance in selecting the optimal treatment. We provided a case study to examine the Statin use on cognitive function to illustrate the use of our proposed method.

Development and Validation of a Treatment Benefit Index to Identify Hospitalized Patients With COVID-19 Who May Benefit From Convalescent Plasma

Importance

Identifying which patients with COVID-19 are likely to benefit from COVID-19 convalescent plasma (CCP) treatment may have a large public health impact.

Objective

To develop an index for predicting the expected relative treatment benefit from CCP compared with treatment without CCP for patients hospitalized for COVID-19 using patients' baseline characteristics.

Design, Setting, and Participants

This prognostic study used data from the COMPILE study, ie, a meta-analysis of pooled individual patient data from 8 randomized clinical trials (RCTs) evaluating CCP vs control in adults hospitalized for COVID-19 who were not receiving mechanical ventilation at randomization. A combination of baseline characteristics, termed the treatment benefit index (TBI), was developed based on 2287 patients in COMPILE using a proportional odds model, with baseline characteristics selected via cross-validation. The TBI was externally validated on 4 external data sets: the Expanded Access Program (1896 participants), a study conducted under Emergency Use Authorization (210 participants), and 2 RCTs (with 80 and 309 participants).

Exposure

Receipt of CCP.

Main Outcomes and Measures

World Health Organization (WHO) 11-point ordinal COVID-19 clinical status scale and 2 derivatives of it (ie, WHO score of 7-10, indicating mechanical ventilation to death, and WHO score of 10, indicating death) at day 14 and day 28 after randomization. Day 14 WHO 11-point ordinal scale was used as the primary outcome to develop the TBI.

Results

A total of 2287 patients were included in the derivation cohort, with a mean (SD) age of 60.3 (15.2) years and 815 (35.6%) women. The TBI provided a continuous gradation of benefit, and, for clinical utility, it was operationalized into groups of expected large clinical benefit (B1; 629 participants in the derivation cohort [27.5%]), moderate benefit (B2; 953 [41.7%]), and potential harm or no benefit (B3; 705 [30.8%]). Patients with preexisting conditions (diabetes, cardiovascular and pulmonary diseases), with blood type A or AB, and at an early COVID-19 stage (low baseline WHO scores) were expected to benefit most, while those without preexisting conditions and at more advanced stages of COVID-19 could potentially be harmed. In the derivation cohort, odds ratios for worse outcome, where smaller odds ratios indicate larger benefit from CCP, were 0.69 (95% credible interval [CrI], 0.48-1.06) for B1, 0.82 (95% CrI, 0.61-1.11) for B2, and 1.58 (95% CrI, 1.14-2.17) for B3. Testing on 4 external datasets supported the validation of the derived TBIs.

Conclusions and Relevance

The findings of this study suggest that the CCP TBI is a simple tool that can quantify the relative benefit from CCP treatment for an individual patient hospitalized with COVID-19 that can be used to guide treatment recommendations. The TBI precision medicine approach could be especially helpful in a pandemic.

Estimation and inference on high-dimensional individualized treatment rule in observational data using split-and-pooled de-correlated score

With the increasing adoption of electronic health records, there is an increasing interest in developing individualized treatment rules, which recommend treatments according to patients' characteristics, from large observational data. However, there is a lack of valid inference procedures for such rules developed from this type of data in the presence of high-dimensional covariates. In this work, we develop a penalized doubly robust method to estimate the optimal individualized treatment rule from high-dimensional data. We propose a split-and-pooled de-correlated score to construct hypothesis tests and confidence intervals. Our proposal adopts the data splitting to conquer the slow convergence rate of nuisance parameter estimations, such as non-parametric methods for outcome regression or propensity models. We establish the limiting distributions of the split-and-pooled de-correlated score test and the corresponding one-step estimator in high-dimensional setting. Simulation and real data analysis are conducted to demonstrate the superiority of the proposed method.

Individualized treatment rules under stochastic treatment cost constraints

Estimation and evaluation of individualized treatment rules have been studied extensively, but real-world treatment resource constraints have received limited attention in existing methods. We investigate a setting in which treatment is intervened upon based on covariates to optimize the mean counterfactual outcome under treatment cost constraints when the treatment cost is random. In a particularly interesting special case, an instrumental variable corresponding to encouragement to treatment is intervened upon with constraints on the proportion receiving treatment. For such settings, we first develop a method to estimate optimal individualized treatment rules. We further construct an asymptotically efficient plug-in estimator of the corresponding average treatment effect relative to a given reference rule.

Non-asymptotic Properties of Individualized Treatment Rules from Sequentially Rule-Adaptive Trials

Learning optimal individualized treatment rules (ITRs) has become increasingly important in the modern era of precision medicine. Many statistical and machine learning methods for learning optimal ITRs have been developed in the literature. However, most existing methods are based on data collected from traditional randomized controlled trials and thus cannot take advantage of the accumulative evidence when patients enter the trials sequentially. It is also ethically important that future patients should have a high probability to be treated optimally based on the updated knowledge so far. In this work, we propose a new design called sequentially rule-adaptive trials to learn optimal ITRs based on the contextual bandit framework, in contrast to the response-adaptive design in traditional adaptive trials. In our design, each entering patient will be allocated with a high probability to the current best treatment for this patient, which is estimated using the past data based on some machine learning algorithm (for example, outcome weighted learning in our implementation). We explore the tradeoff between training and test values of the estimated ITR in single-stage problems by proving theoretically that for a higher probability of following the estimated ITR, the training value converges to the optimal value at a faster rate, while the test value converges at a slower rate. This problem is different from traditional decision problems in the sense that the training data are generated sequentially and are dependent. We also develop a tool that combines martingale with empirical process to tackle the problem that cannot be solved by previous techniques for i.i.d. data. We show by numerical examples that without much loss of the test value, our proposed algorithm can improve the training value significantly as compared to existing methods. Finally, we use a real data study to illustrate the performance of the proposed method.

Multi-Response Based Personalized Treatment Selection with Data from Crossover Designs for Multiple Treatments

In this work we propose a novel method for treatment selection based on individual covariate information when the treatment response is multivariate and data are available from a crossover design. Our method covers any number of treatments and it can be applied for a broad set of models. The proposed method uses a rank aggregation technique to estimate an ordering of treatments based on ranked lists of treatment performance measures such as smooth conditional means and conditional probability of a response for one treatment dominating others. An empirical study demonstrates the performance of the proposed method in finite samples.

Restricted survival benefit with right-censored data

The hazard ratio is widely used to quantify treatment effects. However, it may be difficult to interpret for patients and practitioners, especially when the hazard ratio is not constant over time. Alternative measures of the treatment effects have been proposed such as the difference of the restricted mean survival times, the difference in survival proportions at some fixed follow-up time, or the net chance of a longer survival. In this paper, we propose the restricted survival benefit (RSB), a quantity that can incorporate multiple useful measurements of treatment effects. Hence, it provides a framework for a comprehensive assessment of the treatment effects. We provide estimation and inference procedures for the RSB that accommodate censored survival outcomes, using methods of the inverse-probability-censoring-weighted U -statistic and the jackknife empirical likelihood. We conduct extensive simulation studies to examine the numerical performance of the proposed method, and we analyze data from a randomized Phase III clinical trial (SWOG S0777) using the proposed method.

Identification and inference for subgroups with differential treatment efficacy from randomized controlled trials with survival outcomes through multiple testing

With the uptake of targeted therapies, instead of the "one-fits-all" approach, modern randomized controlled trials (RCTs) often aim to develop treatments that target a subgroup of patients. Motivated by analyzing the Age-Related Eye Disease Study (AREDS) data, a large RCT to study the efficacy of nutritional supplements in delaying the progression of an eye disease, age-related macular degeneration (AMD), we develop a simultaneous inference procedure to identify and infer subgroups with differential treatment efficacy in RCTs with time-to-event outcomes. Specifically, we formulate the multiple testing problem through contrasts and construct their simultaneous confidence intervals, which appropriately control both within- and across-marker multiplicity. Realistic simulations are conducted using real genotype data to evaluate the method performance under various scenarios. The method is then applied to AREDS to assess the efficacy of antioxidants and zinc combination in delaying AMD progression. Multiple gene regions including ESRRB-VASH1 on chromosome 14 have been identified with subgroups showing differential efficacy. We further validate our findings in an independent subsequent RCT, AREDS2, by discovering consistent differential treatment responses in the targeted and non-targeted subgroups identified from AREDS. This multiple-testing-based simultaneous inference approach provides a step forward to confidently identify and infer subgroups in modern drug development.

Risk Minimization from Adaptively Collected Data: Guarantees for Supervised and Policy Learning

Empirical risk minimization (ERM) is the workhorse of machine learning, whether for classification and regression or for off-policy policy learning, but its model-agnostic guarantees can fail when we use adaptively collected data, such as the result of running a contextual bandit algorithm. We study a generic importance sampling weighted ERM algorithm for using adaptively collected data to minimize the average of a loss function over a hypothesis class and provide first-of-their-kind generalization guarantees and fast convergence rates. Our results are based on a new maximal inequality that carefully leverages the importance sampling structure to obtain rates with the good dependence on the exploration rate in the data. For regression, we provide fast rates that leverage the strong convexity of squared-error loss. For policy learning, we provide regret guarantees that close an open gap in the existing literature whenever exploration decays to zero, as is the case for bandit-collected data. An empirical investigation validates our theory.