Inference on Selected Subgroups in Clinical Trials

Modern approaches for evaluating treatment effect heterogeneity from clinical trials and observational data

In this paper, we review recent advances in statistical methods for the evaluation of the heterogeneity of treatment effects (HTE), including subgroup identification and estimation of individualized treatment regimens, from randomized clinical trials and observational studies. We identify several types of approaches using the features introduced in Lipkovich et al (Stat Med 2017;36: 136-196) that distinguish the recommended principled methods from basic methods for HTE evaluation that typically rely on rules of thumb and general guidelines (the methods are often referred to as common practices). We discuss the advantages and disadvantages of various principled methods as well as common measures for evaluating their performance. We use simulated data and a case study based on a historical clinical trial to illustrate several new approaches to HTE evaluation.

Exploratory subgroup identification in the heterogeneous Cox model: A relatively simple procedure

For survival analysis applications we propose a novel procedure for identifying subgroups with large treatment effects, with focus on subgroups where treatment is potentially detrimental. The approach, termed forest search, is relatively simple and flexible. All-possible subgroups are screened and selected based on hazard ratio thresholds indicative of harm with assessment according to the standard Cox model. By reversing the role of treatment one can seek to identify substantial benefit. We apply a splitting consistency criteria to identify a subgroup considered "maximally consistent with harm." The type-1 error and power for subgroup identification can be quickly approximated by numerical integration. To aid inference we describe a bootstrap bias-corrected Cox model estimator with variance estimated by a Jacknife approximation. We provide a detailed evaluation of operating characteristics in simulations and compare to virtual twins and generalized random forests where we find the proposal to have favorable performance. In particular, in our simulation setting, we find the proposed approach favorably controls the type-1 error for falsely identifying heterogeneity with higher power and classification accuracy for substantial heterogeneous effects. Two real data applications are provided for publicly available datasets from a clinical trial in oncology, and HIV.

Finding the best subgroup with differential treatment effect with multiple outcomes

Precision medicine aims to identify specific patient subgroups that may benefit the most from a particular treatment than the whole population. Existing definitions for the best subgroup in subgroup analysis are based on a single outcome and do not consider multiple outcomes; specifically, outcomes of different types. In this article, we introduce a definition for the best subgroup under a multiple-outcome setting with continuous, binary, and censored time-to-event outcomes. Our definition provides a trade-off between the subgroup size and the conditional average treatment effects (CATE) in the subgroup with respect to each of the outcomes while taking the relative contribution of the outcomes into account. We conduct simulations to illustrate the proposed definition. By examining the outcomes of urinary tract infection and renal scarring in the RIVUR clinical trial, we identify a subgroup of children that would benefit the most from long-term antimicrobial prophylaxis.

Inference on tree-structured subgroups with subgroup size and subgroup effect relationship in clinical trials

When multiple candidate subgroups are considered in clinical trials, we often need to make statistical inference on the subgroups simultaneously. Classical multiple testing procedures might not lead to an interpretable and efficient inference on the subgroups as they often fail to take subgroup size and subgroup effect relationship into account. In this paper, built on the selective traversed accumulation rules (STAR), we propose a data-adaptive and interactive multiple testing procedure for subgroups which can take subgroup size and subgroup effect relationship into account under prespecified tree structure. The proposed method is easy-to-implement and can lead to a more interpretable and efficient inference on prespecified tree-structured subgroups. Possible accommodations to post hoc identified tree-structure subgroups are also discussed in the paper. We demonstrate the merit of our proposed method by re-analyzing the panitumumab trial with the proposed method.

A Proposal for Post Hoc Subgroup Analysis in Support of Regulatory Submission

BACKGROUND

In clinical trials, it is not uncommon that the primary analysis fails to achieve the study objective for demonstrating the safety and efficacy of a test treatment under investigation, while a specific sub-population analysis shows a significant positive result. In this case, whether the observed positive sub-population analysis results can be used in support of regulatory submission of the test treatment under investigation is an interesting question to both the investigator(s) and the regulatory medical/statistical reviewers.

METHODS

In this article, several statistical evaluations for confirming the integrity and validity of the observed sub-population analysis results were proposed in support of the regulatory submission. Selection bias caused by looking at one subgroup is adjusted before all statistical evaluations, including reproducibility, consistency between sub-population and the entire population, generalizability between the promising sub-population and other sub-populations, and sensitivity index when there are shifts in mean and/or variability between sub-populations. The multiplicity issue is also addressed in measuring generalizability.

RESULTS

A numerical example of a global (multi-regional) clinical trial was presented for illustration purposes. The choice of applying which estimation approach relies on the scale of test statistics. Recommendations for incorporating statistical evaluations in measuring sub-population analysis are provided. Finally, we proposed possible solutions such as real-world data and real-world evidence for regulatory concerns, which may increase the insufficient power.

CONCLUSION

Sub-population analysis can contribute to regulatory submission if it passes the evaluation. This analysis can also support hypothesis generation and the planning of future clinical trials, though it fails to pass the measurement process.

A savings intervention to reduce men's engagement in HIV risk behaviors: study protocol for a randomized controlled trial

BACKGROUND

In much of eastern and southern Africa, the incidence of HIV and other sexually transmitted infections (STIs) remains high despite the scale-up of promising biomedical and behavioral interventions. Studies have documented the crucial role of transactional sex-the exchange of money, material support, or goods, in sexual relationships-and heavy alcohol use in contributing to men's and women's health outcomes. Existing policy responses to this challenge have largely focused on women, through the provision of pre-exposure prophylaxis (PrEP) or structural interventions such as education subsidies and cash transfers. However, the effectiveness of these interventions has been hindered by the relative lack of interventions and programs targeting men's behavior. We describe the protocol for a study that will test an economic intervention designed to reduce men's engagement in HIV/STI-related risk behaviors in Kenya.

METHODS

We will conduct a randomized controlled trial among income-earning men in Kenya who are aged 18-39 years and self-report alcohol use and engagement in transactional sex. The study will enroll 1500 participants and randomize them to a control group or savings group. The savings group will receive access to a savings account that includes lottery-based incentives to save money regularly, opportunities to develop savings goals/strategies, and text message reminders about their savings goals. The control group will receive basic health education. Over a period of 24 months, we will collect qualitative and quantitative data from participants and a subset of their female partners. Participants will also be tested for HIV and other STIs at baseline, 12, and 24 months.

DISCUSSION

The findings from this study have the potential to address a missing element of HIV/STI prevention efforts in sub-Saharan Africa by promoting upstream and forward-looking behavior and reducing the risk of acquiring HIV/STIs in a high HIV/STI burden setting. If this study is effective, it is an innovative approach that could be scaled up and could have great potential for scientific and public health impact in Kenya.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05385484 . Registered on May 23, 2022.

AN OMNIBUS TEST FOR DETECTION OF SUBGROUP TREATMENT EFFECTS VIA DATA PARTITIONING

Late-stage clinical trials have been conducted primarily to establish the efficacy of a new treatment in an intended population. A corollary of population heterogeneity in clinical trials is that a treatment might be effective for one or more subgroups, rather than for the whole population of interest. As an example, the phase III clinical trial of panitumumab in metastatic colorectal cancer patients failed to demonstrate its efficacy in the overall population, but a subgroup associated with tumor KRAS status was found to be promising (Peeters et al. (Am. J. Clin. Oncol. 28 (2010) 4706-4713)). As we search for such subgroups via data partitioning based on a large number of biomarkers, we need to guard against inflated type I error rates due to multiple testing. Commonly-used multiplicity adjustments tend to lose power for the detection of subgroup treatment effects. We develop an effective omnibus test to detect the existence of, at least, one subgroup treatment effect, allowing a large number of possible subgroups to be considered and possibly censored outcomes. Applied to the panitumumab trial data, the proposed test would confirm a significant subgroup treatment effect. Empirical studies also show that the proposed test is applicable to a variety of outcome variables and maintains robust statistical power.

A Machine-Learning Approach for Estimating Subgroup- and Individual-Level Treatment Effects: An Illustration Using the 65 Trial

Personalizing treatment recommendations or guidelines requires evidence about the heterogeneity of treatment effects (HTE). Machine-learning (ML) approaches can explore HTE by considering many covariates, including complex interactions between them. Causal ML approaches can avoid overfitting, which arises when the same dataset is used to select covariate by treatment interaction terms as to make inferences and reduce reliance on the correct specification of fixed parametric models. We investigate causal forests (CF), a ML method based on modified decision trees that can estimate subgroup- and individual-level treatment effects, without requiring correct prespecification of the effect model. We consider CF alongside parametric approaches for estimating HTE, within the 65 Trial, which evaluates the effect of a permissive hypotension strategy versus usual care on 90-d mortality for critically ill patients aged 65 y or older with vasodilatory hypotension. Here, the CF approach provides similar estimates of treatment effectiveness for prespecified and post hoc subgroups to the parametric approach, and the results of a test for overall HTE show weak evidence of heterogeneity. The CF estimates of individual-level treatment effects, the expected effects of treatment for individuals in subpopulations defined by their covariates, suggest that the permissive hypotension strategy is expected to reduce 90-d mortality for 98.7% of patients but with 95% confidence intervals that include zero for 71.6% of patients. A ML approach is then used to assess the patient characteristics associated with these individual-level effects, and to help target future research that can identify those patient subgroups for whom the intervention is most effective.

Multiply robust subgroup analysis based on a single-index threshold linear marginal model for longitudinal data with dropouts

Identifying subpopulations that may be sensitive to the specific treatment is an important step toward precision medicine. On the other hand, longitudinal data with dropouts is common in medical research, and subgroup analysis for this data type is still limited. In this paper, we consider a single-index threshold linear marginal model, which can be used simultaneously to identify subgroups with differential treatment effects based on linear combination of the selected biomarkers, estimate the treatment effects in different subgroups based on regression coefficients, and test the significance of the difference in treatment effects based on treatment-subgroup interaction. The regression parameters are estimated by solving a penalized smoothed generalized estimating equation and the selection bias caused by missingness is corrected by a multiply robust weighting matrix, which allows multiple missingness models to be taken account into estimation. The proposed estimator remains consistent when any model for missingness is correctly specified. Under regularity conditions, the asymptotic normality of the estimator is established. Simulation studies confirm the desirable finite-sample performance of the proposed method. As an application, we analyze the data from a clinical trial on pancreatic cancer.

Improving the assessment of the probability of success in late stage drug development

There are several steps to confirming the safety and efficacy of a new medicine. A sequence of trials, each with its own objectives, is usually required. Quantitative risk metrics can be useful for informing decisions about whether a medicine should transition from one stage of development to the next. To obtain an estimate of the probability of regulatory approval, pharmaceutical companies may start with industry-wide success rates and then apply to these subjective adjustments to reflect program-specific information. However, this approach lacks transparency and fails to make full use of data from previous clinical trials. We describe a quantitative Bayesian approach for calculating the probability of success (PoS) at the end of phase II which incorporates internal clinical data from one or more phase IIb studies, industry-wide success rates, and expert opinion or external data if needed. Using an example, we illustrate how PoS can be calculated accounting for differences between the phase II data and future phase III trials, and discuss how the methods can be extended to accommodate accelerated drug development pathways.

Causal mediation analysis with latent subgroups

In biomedical studies, the causal mediation effect might be heterogeneous across individuals in the study population due to each study subject's unique characteristics. While individuals' mediation effects may differ from each other, it is often reasonable and more interpretable to assume that individuals belong to several distinct latent subgroups with similar attributes. In this article, we first show that the subgroup-specific mediation effect can be identified under the group-specific sequential ignorability assumptions. Then, we propose a simple mixture modeling approach to account for the latent subgroup structure where each mixture component corresponds to one latent subgroup in the linear structural equation model framework. Model parameters can be estimated using the standard expectation-maximization (EM) algorithm. Each individual's subgroup membership can be inferred based on the posterior probability. We propose to use the singular Bayesian information criterion to consistently select the number of latent subgroups by recognizing that the Fisher information matrix for mixture models might be singular. We then propose to use nonparametric bootstrap method to compute standard errors and confidence intervals. We conducted simulation studies to evaluate the empirical performance of our proposed method named iMed. Finally, we reanalyzed a DNA methylation data set from the Normative Aging Study and found that the mediation effects of two well-documented DNA methylation CpG sites are heterogeneous across two latent subgroups in the causal pathway from smoking behavior to lung function. We also developed an R package iMed for public use.