Adaptive non-inferiority margins under observable non-constancy

Estimating counterfactual placebo HIV incidence in HIV prevention trials without placebo arms based on markers of HIV exposure

INTRODUCTION

Developing alternative approaches to evaluating absolute efficacy of new HIV prevention interventions is a priority, as active-controlled designs, whereby individuals without HIV are randomized to the experimental intervention or an active control known to be effective, are increasing. With this design, however, the efficacy of the experimental intervention to prevent HIV acquisition relative to placebo cannot be evaluated directly.

METHODS

One proposed approach to estimate absolute prevention efficacy is to use an HIV exposure marker, such as incident rectal gonorrhea, to infer counterfactual placebo HIV incidence. We formalize a statistical framework for this approach, specify working regression and likelihood-based estimation approaches, lay out three assumptions under which valid inference can be achieved, evaluate finite-sample performance, and illustrate the approach using a recent active-controlled HIV prevention trial.

RESULTS

We find that in finite samples and under correctly specified assumptions accurate and precise estimates of counterfactual placebo incidence and prevention efficacy are produced. Based on data from the DISCOVER trial in men and transgender women who have sex with men, and assuming correctly specified assumptions, the estimated prevention efficacy for tenofovir alafenamide plus emtricitabine is 98.1% (95% confidence interval: 96.4%-99.4%) using the working model approach and 98.1% (95% confidence interval: 96.4%-99.7%) using the likelihood-based approach.

CONCLUSION

Careful assessment of the underlying assumptions, study of their violation, evaluation of the approach in trials with placebo arms, and advancement of improved exposure markers are needed before the HIV exposure marker approach can be relied upon in practice.

The Smooth Away From Expected (SAFE) non-inferiority frontier: theory and implementation with an application to the D3 trial

BACKGROUND

In a non-inferiority trial, the choice of margin depends on the expected control event risk. If the true risk differs from expected, power and interpretability of results can be affected. A non-inferiority frontier pre-specifies an appropriate non-inferiority margin for each value of control event risk. D3 is a non-inferiority trial comparing two treatment regimens in children living with HIV, designed assuming a control event risk of 12%, a non-inferiority margin of 10%, 80% power and a significance level (α) of 0.025. We consider approaches to choosing and implementing a frontier for this already funded trial, where changing the sample size substantially would be difficult.

METHODS

In D3, we fix the non-inferiority margin at 10%, 8% and 5% for control event risks of ≥9%, 5% and 1%, respectively. We propose four frontiers which fit these fixed points, including a Smooth Away From Expected (SAFE) frontier. Analysis approaches considered are as follows: using the pre-specified significance level (α=0.025); always using a reduced significance level (to achieve α≤0.025 across control event risks); reducing significance levels only when the control event risk differs significantly from expected (control event risk <9%); and using a likelihood ratio test. We compare power and type 1 error for SAFE with other frontiers.

RESULTS

Changing the significance level only when the control event risk is <9% achieves approximately nominal (<3%) type I error rate and maintains reasonable power for control event risks between 1 and 15%. The likelihood ratio test method performs similarly, but the results are more complex to present. Other analysis methods lead to either inflated type 1 error or badly reduced power. The SAFE frontier gives more interpretable results with low control event risks than other frontiers (i.e. it uses more reasonable non-inferiority margins). Other frontiers do not achieve power close (i.e. within 1%) to SAFE across the range of likely control event risks while controlling type I error.

CONCLUSIONS

The SAFE non-inferiority frontier will be used in D3, and the non-inferiority margin and significance level will be modified if the control event risk is lower than expected. This ensures results will remain interpretable if design assumptions are incorrect, while achieving similar power. A similar approach could be considered for other non-inferiority trials where the control event risk is uncertain.

Counterfactual estimation of efficacy against placebo for novel PrEP agents using external trial data: example of injectable cabotegravir and oral PrEP in women

INTRODUCTION

Multiple antiretroviral agents have demonstrated efficacy for human immunodeficiency virus (HIV) pre-exposure prophylaxis (PrEP). As a result, clinical trials of novel agents have transitioned from placebo- to active-controlled designs; however, active-controlled trials do not provide an estimate of efficacy versus no use of PrEP. Counterfactual placebo comparisons using other data sources could be employed to provide this information.

METHODS

We compared the active-controlled study (HPTN 084) of injectable cabotegravir (CAB-LA) versus daily oral emtricitabine/tenofovir disoproxil fumarate (FTC/TDF) among women from seven countries in Africa to three external, contemporaneous randomized HIV prevention trials from which we constructed counterfactual placebo estimates. We used direct standardization via analysis weights to achieve the same distribution of person-years between the external study and HPTN 084, across strata predictive of HIV risk (country and selected risk covariates). We estimated prevention efficacy against a counterfactual placebo to provide information on the use of CAB-LA and FTC/TDF compared to no intervention. We compared the counterfactual placebo findings for FTC/TDF to previous placebo-controlled trials, adjusted for observed adherence to daily pills.

RESULTS

Distribution of age and baseline prevalence of gonorrhoea and chlamydia were similar among matched counterfactual placebo and observed HPTN 084 arms after standardization. Counterfactual estimates of CAB-LA versus placebo in all three settings showed a consistent risk reduction of 93%-94%, with lower bounds of the confidence intervals above 72%. Observed adherence (quantifiable tenofovir in plasma) in HPTN 084 was 54%-56%, and estimated efficacy of daily oral FTC/TDF against a counterfactual placebo was consistent with a predicted risk reduction of 39%-40% for this level of daily pill use.

CONCLUSIONS

Counterfactual placebo rates of HIV acquisition derived from external trial data in similar locations and time can be used to support estimates of placebo-based efficacy of a novel HIV prevention agent. External trial data must be standardized to be representative of the clinical trial cohort testing the novel HIV prevention agent, accounting for confounders.

Estimating the impact of HIV PrEP regimens containing long-acting injectable cabotegravir or daily oral tenofovir disoproxil fumarate/emtricitabine among men who have sex with men in the United States: a mathematical modelling study for HPTN 083

Background

The HPTN 083 trial demonstrated superiority of HIV pre-exposure prophylaxis (PrEP) containing long-acting injectable cabotegravir (CAB) to daily oral tenofovir disoproxil fumarate/emtricitabine (TDF/FTC) among men who have sex with men (MSM). We compared the potential population-level impact of TDF/FTC and CAB among MSM in Atlanta, Georgia.

Methods

An MSM HIV transmission model was calibrated to Atlanta-specific data on HIV prevalence and PrEP usage (percentage of uninfected MSM on PrEP), assuming only PrEP-indicated MSM used PrEP. CAB effectiveness (efficacy × adherence) of 91% was estimated using data from HPTN 083 and previous TDF/FTC trials. We estimated HIV infections averted over 5/10 years if TDF/FTC use were maintained, or if all TDF/FTC users switched to CAB in January 2022 (vs. no PrEP or continued TDF/FTC use). CAB scenarios with 10%/20% more users were also considered. Progress towards Ending the HIV Epidemic (EHE) goals (75%/90% fewer HIV infections in 2025/2030 vs. 2017) was estimated.

Findings

We predicted TDF/FTC at current usage (∼28%) would avert 36.3% of new HIV infections (95% credible interval 25.6-48.7%) among all Atlanta MSM over 2022-2026 vs. no PrEP. Switching to CAB with similar usage may prevent 44.6% (33.2-56.6%) infections vs. no PrEP and 11.9% (5.2-20.2%) infections vs. continued TDF/FTC. Increasing CAB usage 20% could increase the incremental impact over TDF/FTC to 30.0% over 2022-2026, getting ∼60% towards reaching EHE goals (47%/54% fewer infections in 2025/2030). Reaching the 2030 EHE goal would require 93% CAB usage.

Interpretation

If CAB effectiveness were like HPTN 083, CAB could prevent more infections than TDF/FTC at similar usage. Increased CAB usage could contribute substantially towards reaching EHE goals, but the usage required to meet EHE goals is unrealistic.

Funding

NIH, MRC.

Evaluating group-sequential non-inferiority clinical trials following interim stopping: The HIV Prevention Trials Network 083 trial

BACKGROUND/AIMS

The HIV Prevention Trials Network 083 trial was a group-sequential non-inferiority trial designed to compare HIV incidence under a novel experimental regimen for HIV prevention, long-acting injectable cabotegravir, with an active-control regimen of daily oral tenofovir disoproxil fumarate/emtricitabine (brand name Truvada). In March of 2020, just as the trial had completed enrollment, the COVID-19 pandemic threatened to prevent trial participants from attending study visits and obtaining study medication, motivating the study team to update the interim monitoring plan. The Data and Safety Monitoring Board subsequently stopped the trial at the first interim review due to strong early evidence of efficacy.

METHODS

Here we describe some unique aspects of the trial's design, monitoring, analysis, and interpretation. We illustrate the importance of computing point estimates, confidence intervals, and p values based on the sampling distribution induced by sequential monitoring.

RESULTS

Accurate analysis, decision-making and interpretation of trial results rely on pre-specification of a stopping boundary, including the scale on which the stopping rule will be implemented, the specific test statistics to be calculated, and how the boundary will be adjusted if the available information fraction at interim review is different from planned. After appropriate adjustment for the sampling distribution and overrun, the HIV Prevention Trials Network 083 trial provided strong evidence that the experimental regimen was superior to the active control.

CONCLUSIONS

For the HIV Prevention Trials Network 083 trial, the difference between corrected inferential statistics and naive results was quite small-as will often be the case-nevertheless, it is appropriate to report and publish the most accurate and unbiased statistical results.

Conversion of non-inferiority margin from hazard ratio to restricted mean survival time difference using data from multiple historical trials

The restricted mean survival time measure has gained a lot of interests for designing and analyzing oncology trials with time-to-event endpoints due to its intuitive clinical interpretation and potentially high statistical power. In the non-inferiority trial literature, restricted mean survival time has been used as an alternative measure for reanalyzing a completed trial, which was originally designed and analyzed based on traditional proportional hazard model. However, the reanalysis procedure requires a conversion from the non-inferiority margin measured in hazard ratio to a non-inferiority margin measured by restricted mean survival time difference. An existing conversion method assumes a Weibull distribution for the population survival time of the historical active control group under the proportional hazard assumption using data from a single trial. In this article, we develop a methodology for non-inferiority margin conversion when data from multiple historical active control studies are available, and introduce a Kaplan-Meier estimator-based method for the non-inferiority margin conversion to relax the parametric assumption. We report extensive simulation studies to examine the performances of proposed methods under the Weibull data generative models and a piecewise-exponential data generative model that mimic the tumor recurrence and survival characteristics of advanced colon cancer. This work is motivated to achieve non-inferiority margin conversion, using historical patient-level data from a large colon cancer clinical database, to reanalyze an internationally collaborated non-inferiority study that evaluates 6-month versus 3-month duration of adjuvant chemotherapy in stage III colon cancer patients.

Ethical issues in establishing the efficacy and safety of long-acting injectable pre-exposure prophylaxis for HIV prevention: the HPTN 083 trial

Two multinational clinical trials have shown safety and efficacy of long-acting injectable cabotegravir for HIV pre-exposure prophylaxis (PrEP). These results will alter the landscape of HIV prevention and related research. Nevertheless, designing and conducting this research involved several ethical issues. This Viewpoint describes how we managed ethical issues over the duration of one of these trials (HPTN 083). Specifically, we discuss the rationale for pursuing a long-acting injectable agent in the presence of effective oral PrEP, trial design choices, site selection and local standards of prevention, data monitoring and early stopping, effects of the COVID-19 pandemic, post-trial access, and assessment of long-term safety.

Using the adherence-efficacy relationship of emtricitabine and tenofovir disoproxil fumarate to calculate background hiv incidence: a secondary analysis of a randomized, controlled trial

INTRODUCTION

Randomized trials of new agents for HIV pre-exposure prophylaxis (PrEP) compare against emtricitabine and tenofovir disoproxil fumarate (F/TDF), without a placebo group. We used the well-characterized adherence-efficacy relationship for F/TDF to back-calculate the (non-PrEP) counterfactual background HIV incidence (bHIV) in a randomized trial of a novel PrEP agent and estimate comparative efficacy (to counterfactual bHIV).

METHODS

The DISCOVER trial (ClinicalTrials.gov: NCT02842086) randomized 5387 men who have sex with men (MSM) and transgender women who have sex with men and demonstrated non-inferiority of emtricitabine and tenofovir alafenamide (F/TAF) to F/TDF (HIV incidence rate ratio [IRR] 0·47, 95% CI: 0·19 to 1.15). Tenofovir diphosphate (TFV-DP) levels in dried blood spots (DBS) were assessed for all diagnosed with HIV and in a random 10% of the cohort. We used a Bayesian model with a diffuse prior distribution, derived from established data relating tenofovir diphosphate levels to HIV prevention efficacy. This prior, combined with the F/TDF seroconversion rate and tenofovir diphosphate levels in DISCOVER, yielded Bayesian inferences on the counterfactual bHIV.

RESULTS

There were six versus 11 postbaseline HIV infections (0.14 vs. 0.25/100 person-years [PY]) on F/TAF and F/TDF respectively. Of the 11 on F/TDF, 10 had low, none had medium and one had high tenofovir diphosphate levels; among HIV-negative controls, 5% of the person-time years had low, 9% had medium and 86% had high TFV-DP levels. A non-informative prior distribution for counterfactual bHIV, combined with the prior for TFV-DP level-efficacy relationship, yielded a posterior counterfactual bHIV of 3·4 infections/100 PY (0.80 Bayesian credible interval [CrI] 1·9 to 5·9), which suggests a median HIV efficacy of 96% (0.95 CrI [88% to 99%]) for F/TAF and 93% (0.95 CrI [87% to 96%]) for F/TDF compared to bHIV.

CONCLUSIONS

Based on the established connection of drug concentrations to PrEP prevention efficacy, a Bayesian framework can be used to estimate a synthetic non-PrEP control group in randomized, active-controlled PrEP trials that include a F/TDF-comparator group.

Sample size calculation for active-arm trial with counterfactual incidence based on recency assay

Objectives

The past decade has seen tremendous progress in the development of biomedical agents that are effective as pre-exposure prophylaxis (PrEP) for HIV prevention. To expand the choice of products and delivery methods, new medications and delivery methods are under development. Future trials of non-inferiority, given the high efficacy of ARV-based PrEP products as they become current or future standard of care, would require a large number of participants and long follow-up time that may not be feasible. This motivates the construction of a counterfactual estimate that approximates incidence for a randomized concurrent control group receiving no PrEP.

Methods

We propose an approach that is to enroll a cohort of prospective PrEP users and aug-ment screening for HIV with laboratory markers of duration of HIV infection to indicate recent infections. We discuss the assumptions under which these data would yield an estimate of the counterfactual HIV incidence and develop sample size and power calculations for comparisons to incidence observed on an investigational PrEP agent.

Results

We consider two hypothetical trials for men who have sex with men (MSM) and transgender women (TGW) from different regions and young women in sub-Saharan Africa. The calculated sample sizes are reasonable and yield desirable power in simulation studies.

Conclusions

Future one-arm trials with counterfactual placebo incidence based on a recency assay can be conducted with reasonable total screening sample sizes and adequate power to determine treatment efficacy.

A Bayesian averted infection framework for PrEP trials with low numbers of HIV infections: application to the results of the DISCOVER trial

Trials of candidate agents for HIV pre-exposure prophylaxis (PrEP) might randomly assign participants to be given a new PrEP agent or oral coformulated tenofovir disoproxil fumarate plus emtricitabine. This design presents unique challenges in interpretation. First, with two active arms, HIV incidence might be low. Second, the effectiveness of tenofovir disoproxil fumarate plus emtricitabine varies across populations; thus, similar HIV incidence between groups could be consistent with a wide range of effectiveness for the new PrEP. We propose a two-part approach to trial results. First, we use Bayesian methods to incorporate assumptions about the background incidence of HIV in the trial in the absence of PrEP, possibly augmented by external data. On the basis of the estimated background incidence, we estimate and compare the number of averted (or prevented) HIV infections in each of the two trial groups, calculating the averted infections ratio. We apply these methods to a completed trial of tenofovir alafenamide plus emtricitabine for PrEP. Our framework shows that leveraging external information to estimate averted infections and the averted infections ratio enhances the efficiency and interpretation of active-controlled PrEP trials.

Handling an uncertain control group event risk in non-inferiority trials: non-inferiority frontiers and the power-stabilising transformation

Background

Non-inferiority trials are increasingly used to evaluate new treatments that are expected to have secondary advantages over standard of care, but similar efficacy on the primary outcome. When designing a non-inferiority trial with a binary primary outcome, the choice of effect measure for the non-inferiority margin (e.g. risk ratio or risk difference) has an important effect on sample size calculations; furthermore, if the control event risk observed is markedly different from that assumed, the trial can quickly lose power or the results become difficult to interpret.

Methods

We propose a new way of designing non-inferiority trials to overcome the issues raised by unexpected control event risks. Our proposal involves using clinical judgement to specify a ‘non-inferiority frontier’, i.e. a curve defining the most appropriate non-inferiority margin for each possible value of control event risk. Existing trials implicitly use frontiers defined by a fixed risk ratio or a fixed risk difference. We discuss their limitations and propose a fixed arcsine difference frontier, using the power-stabilising transformation for binary outcomes, which may better represent clinical judgement. We propose and compare three ways of designing a trial using this frontier: testing and reporting on the arcsine scale; testing on the arcsine scale but reporting on the risk difference or risk ratio scale; and modifying the margin on the risk difference or risk ratio scale after observing the control event risk according to the power-stabilising frontier.

Results

Testing and reporting on the arcsine scale leads to results which are challenging to interpret clinically. For small values of control event risk, testing on the arcsine scale and reporting results on the risk difference scale produces confidence intervals at a higher level than the nominal one or non-inferiority margins that are slightly smaller than those back-calculated from the power-stabilising frontier alone. However, working on the arcsine scale generally requires a larger sample size compared to the risk difference scale. Therefore, working on the risk difference scale, modifying the margin after observing the control event risk, might be preferable, as it requires a smaller sample size. However, this approach tends to slightly inflate type I error rate; a solution is to use a slightly lower significance level for testing, although this modestly reduces power. When working on the risk ratio scale instead, the same approach based on the modification of the margin leads to power levels above the nominal one, maintaining type I error under control.

Conclusions

Our proposed methods of designing non-inferiority trials using power-stabilising non-inferiority frontiers make trial design more resilient to unexpected values of the control event risk, at the only cost of requiring somewhat larger sample sizes when the goal is to report results on the risk difference scale.

Advancing Novel PrEP Products - Alternatives to Non-Inferiority

With the scale-up of HIV pre-exposure prophylaxis (PrEP) with tenofovir (TDF) with or without emtricitabine (FTC), we have entered an era of highly effective HIV prevention with a growing pipeline of potential products to be studied. These studies are likely to be randomized trials with an oral TDF/FTC control arm. These studies require comparison of incident infections and can be time and resource intensive. Conventional approaches for design and analysis active controlled trial can lead to very large sample sizes. We demonstrate the important of assumptions about background infections for interpreting trial results and suggest alternative criteria for demonstrating the efficacy and effectiveness of potential PrEP agents.

The Connection between the Averted Infections Ratio and the Rate Ratio in Active-control Trials of Pre-exposure Prophylaxis Agents

The design and analysis of active-control trials to evaluate experimental HIV pre-exposure prophylaxis (PrEP) agents pose serious statistical challenges. We recently proposed a new outcome measure, the averted infections ratio (AIR) – the proportion of infections that would be averted by using the experimental agent rather than the control agent (compared to no intervention). The main aim of the current paper is to examine the mathematical connection between AIR and the HIV incidence rate ratio, the standard outcome measure. We also consider the sample size implications of the choice of primary outcome measure and explore the connection between effectiveness and efficacy under a simplified model of adherence.