An approach to explore for a sweet spot in randomized trials

Impact of the PATH Statement on Analysis and Reporting of Heterogeneity of Treatment Effect in Clinical Trials: A Scoping Review

Background

The Predictive Approaches to Treatment Effect Heterogeneity (PATH) Statement provides guidance for using predictive modeling to identify differences (i.e., heterogeneity) in treatment effects (benefits and harms) among participants in randomized clinical trials (RCTs). It distinguished risk modeling, which uses a multivariable model to predict risk of trial outcome(s) and then examines treatment effects within strata of predicted risk, from effect modeling, which predicts trial outcomes using models that include treatment, individual participant characteristics and interactions of treatment with selected characteristics.

Purpose

To describe studies of heterogeneous treatment effects (HTE) that use predictive modeling in RCT data and cite the PATH Statement.

Data Sources

The Cited By functions in PubMed, Google Scholar, Web of Science and SCOPUS databases (Jan 7, 2020 - June 5, 2023).

Study Selection

42 reports presenting 45 predictive models.

Data Extraction

Double review with adjudication to identify risk and effect modeling and examine consistency with Statement consensus statements. Credibility of HTE findings was assessed using criteria adapted from the Instrument to assess Credibility of Effect Modification Analyses (ICEMAN). Clinical importance of credible HTE findings was also assessed.

Data Synthesis

The numbers of reports, especially risk modeling reports, increased year-on-year. Consistency with consensus statements was high, except for two: only 15 of 32 studies with positive overall findings included a risk model; and most effect models explored many candidate covariates with little prior evidence for effect modification. Risk modeling was more likely than effect modeling to identify both credible HTE (14/19 vs 5/26) and clinically important HTE (10/19 vs 4/26).

Limitations

Risk of reviewer bias: reviewers assessing credibility and clinical importance were not blinded to adherence to PATH recommendations.

Conclusions

The PATH Statement appears to be influencing research practice. Risk modeling often uncovered clinically important HTE; effect modeling was more often exploratory.

Testing for a Sweet Spot in Randomized Trials

Introduction

Randomized trials recruit diverse patients, including some individuals who may be unresponsive to the treatment. Here we follow up on prior conceptual advances and introduce a specific method that does not rely on stratification analysis and that tests whether patients in the intermediate range of disease severity experience more relative benefit than patients at the extremes of disease severity (sweet spot).

Methods

We contrast linear models to sigmoidal models when describing associations between disease severity and accumulating treatment benefit. The Gompertz curve is highlighted as a specific sigmoidal curve along with the Akaike information criterion (AIC) as a measure of goodness of fit. This approach is then applied to a matched analysis of a published landmark randomized trial evaluating whether implantable defibrillators reduce overall mortality in cardiac patients (n = 2,521).

Results

The linear model suggested a significant survival advantage across the spectrum of increasing disease severity (β = 0.0847, P < 0.001, AIC = 2,491). Similarly, the sigmoidal model suggested a significant survival advantage across the spectrum of disease severity (α = 93, β = 4.939, γ = 0.00316, P < 0.001 for all, AIC = 1,660). The discrepancy between the 2 models indicated worse goodness of fit with a linear model compared to a sigmoidal model (AIC: 2,491 v. 1,660, P < 0.001), thereby suggesting a sweet spot in the midrange of disease severity. Model cross-validation using computational statistics also confirmed the superior goodness of fit of the sigmoidal curve with a concentration of survival benefits for patients in the midrange of disease severity.

Conclusion

Systematic methods are available beyond simple stratification for identifying a sweet spot according to disease severity. The approach can assess whether some patients experience more relative benefit than other patients in a randomized trial.