Estimating vaccine efficacy during open-label follow-up of COVID-19 vaccine trials based on population-level surveillance data

While rapid development and roll out of COVID-19 vaccines is necessary in a pandemic, the process limits the ability of clinical trials to assess longer-term vaccine efficacy. We leveraged COVID-19 surveillance data in the U.S. to evaluate vaccine efficacy in U.S. Government-funded COVID-19 vaccine efficacy trials with a three-step estimation process. First, we used a compartmental epidemiological model informed by county-level surveillance data, a "population model", to estimate SARS-CoV-2 incidence among the unvaccinated. Second, a "cohort model" was used to adjust the population SARS-CoV-2 incidence to the vaccine trial cohort, taking into account individual participant characteristics and the difference between SARS-CoV-2 infection and COVID-19 disease. Third, we fit a regression model estimating the offset between the cohort-model-based COVID-19 incidence in the unvaccinated with the placebo-group COVID-19 incidence in the trial during blinded follow-up. Counterfactual placebo COVID-19 incidence was estimated during open-label follow-up by adjusting the cohort-model-based incidence rate by the estimated offset. Vaccine efficacy during open-label follow-up was estimated by contrasting the vaccine group COVID-19 incidence with the counterfactual placebo COVID-19 incidence. We documented good performance of the methodology in a simulation study. We also applied the methodology to estimate vaccine efficacy for the two-dose AZD1222 COVID-19 vaccine using data from the phase 3 U.S. trial (ClinicalTrials.gov # NCT04516746). We estimated AZD1222 vaccine efficacy of 59.1% (95% uncertainty interval (UI): 40.4%-74.3%) in April, 2021 (mean 106 days post-second dose), which reduced to 35.7% (95% UI: 15.0%-51.7%) in July, 2021 (mean 198 days post-second-dose). We developed and evaluated a methodology for estimating longer-term vaccine efficacy. This methodology could be applied to estimating counterfactual placebo incidence for future placebo-controlled vaccine efficacy trials of emerging pathogens with early termination of blinded follow-up, to active-controlled or uncontrolled COVID-19 vaccine efficacy trials, and to other clinical endpoints influenced by vaccination.

Estimating counterfactual placebo HIV incidence in HIV prevention trials without placebo arms based on markers of HIV exposure

INTRODUCTION

Developing alternative approaches to evaluating absolute efficacy of new HIV prevention interventions is a priority, as active-controlled designs, whereby individuals without HIV are randomized to the experimental intervention or an active control known to be effective, are increasing. With this design, however, the efficacy of the experimental intervention to prevent HIV acquisition relative to placebo cannot be evaluated directly.

METHODS

One proposed approach to estimate absolute prevention efficacy is to use an HIV exposure marker, such as incident rectal gonorrhea, to infer counterfactual placebo HIV incidence. We formalize a statistical framework for this approach, specify working regression and likelihood-based estimation approaches, lay out three assumptions under which valid inference can be achieved, evaluate finite-sample performance, and illustrate the approach using a recent active-controlled HIV prevention trial.

RESULTS

We find that in finite samples and under correctly specified assumptions accurate and precise estimates of counterfactual placebo incidence and prevention efficacy are produced. Based on data from the DISCOVER trial in men and transgender women who have sex with men, and assuming correctly specified assumptions, the estimated prevention efficacy for tenofovir alafenamide plus emtricitabine is 98.1% (95% confidence interval: 96.4%-99.4%) using the working model approach and 98.1% (95% confidence interval: 96.4%-99.7%) using the likelihood-based approach.

CONCLUSION

Careful assessment of the underlying assumptions, study of their violation, evaluation of the approach in trials with placebo arms, and advancement of improved exposure markers are needed before the HIV exposure marker approach can be relied upon in practice.

Study design approaches for future active-controlled HIV prevention trials

Objectives

Vigorous discussions are ongoing about future efficacy trial designs of candidate human immunodeficiency virus (HIV) prevention interventions. The study design challenges of HIV prevention interventions are considerable given rapid evolution of the prevention landscape and evidence of multiple modalities of highly effective products; future trials will likely be 'active-controlled', i.e., not include a placebo arm. Thus, novel design approaches are needed to accurately assess new interventions against these highly effective active controls.

Methods

To discuss active control design challenges and identify solutions, an initial virtual workshop series was hosted and supported by the International AIDS Enterprise (October 2020-March 2021). Subsequent symposia discussions continue to advance these efforts. As the non-inferiority design is an important conceptual reference design for guiding active control trials, we adopt several of its principles in our proposed design approaches.

Results

We discuss six potential study design approaches for formally evaluating absolute prevention efficacy given data from an active-controlled HIV prevention trial including using data from: 1) a registrational cohort, 2) recency assays, 3) an external trial placebo arm, 4) a biomarker of HIV incidence/exposure, 5) an anti-retroviral drug concentration as a mediator of prevention efficacy, and 6) immune biomarkers as a mediator of prevention efficacy.

Conclusions

Our understanding of these proposed novel approaches to future trial designs remains incomplete and there are many future statistical research needs. Yet, each of these approaches, within the context of an active-controlled trial, have the potential to yield reliable evidence of efficacy for future biomedical interventions.

Control groups for HIV prevention efficacy trials: what does the future hold?

PURPOSE OF REVIEW

Ending the HIV epidemic will require the development of additional effective immune-mediated and nonimmune-mediated means of HIV prevention. Evaluating novel interventions requires large, controlled trials demonstrating efficacy. Recent advances in the field of HIV prevention necessitate new approaches to efficacy trial design.

RECENT FINDINGS

Three classes of efficacy trial designs are possible: standard of prevention-controlled trials, active-controlled trials, and active-controlled trials augmented with external control data. Recent experience with these approaches provides lessons on considerations around and success of the designs. Additional experience and development is needed for the augmented active-controlled trial design.

SUMMARY

Efficacy trials of new HIV prevention interventions are feasible but require careful consideration, given the complexity and dynamic state of the prevention field. While standard of prevention-controlled efficacy trials are reasonable approaches for HIV vaccine and monoclonal antibody efficacy trials, trials of new antiretroviral agents may require active-controlled designs.

Efficacy of Messenger RNA-1273 Against Severe Acute Respiratory Syndrome Coronavirus 2 Acquisition in Young Adults From March to December 2021

Background

The efficacy of messenger RNA (mRNA)-1273 against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection is not well defined, particularly among young adults.

Methods

Adults aged 18-29 years with no known history of SARS-CoV-2 infection or prior vaccination for coronavirus disease 2019 (COVID-19) were recruited from 44 US sites from 24 March to 13 September 2021 and randomized 1:1 to immediate vaccination (receipt of 2 doses of mRNA-1273 vaccine at months 0 and 1) or the standard of care (receipt of COVID-19 vaccine). Randomized participants were followed up for SARS-CoV-2 infection measured by nasal swab testing and symptomatic COVID-19 measured by nasal swab testing plus symptom assessment and assessed for the primary efficacy outcome. A vaccine-declined observational group was also recruited from 16 June to 8 November 2021 and followed up for SARS-CoV-2 infection as specified for the randomized participants.

Results

The study enrolled 1149 in the randomized arms and 311 in the vaccine-declined group and collected >122 000 nasal swab samples. Based on randomized participants, the efficacy of 2 doses of mRNA-1273 vaccine against SARS-CoV-2 infection was 52.6% (95% confidence interval, -14.1% to 80.3%), with the majority of infections due to the Delta variant. Vaccine efficacy against symptomatic COVID-19 was 71.0% (95% confidence interval, -9.5% to 92.3%). Precision was limited owing to curtailed study enrollment and off-study vaccination censoring. The incidence of SARS-CoV-2 infection in the vaccine-declined group was 1.8 times higher than in the standard-of-care group.

Conclusions

mRNA-1273 vaccination reduced the incidence of SARS-CoV-2 infection from March to September 2021, but vaccination was only one factor influencing risk.

Clinical Trials Registration

NCT04811664.

Perspectives on Design Approaches for HIV Prevention Efficacy Trials

The challenge of designing future HIV prevention efficacy trials in a rapidly evolving HIV prevention landscape was explored through a series of virtual stakeholder's engagement meetings convened online between October 2020 and April 2021. A broad array of stakeholders from the HIV prevention research community reviewed current trial designs and lessons learned, explored issues specific to unique product classes, and concluded with specialist-focused examinations of statistical design concepts and the importance of community engagement in research. The aim was to reflect on current approaches and evaluate new trial design approaches for evaluating efficacy of a candidate prevention strategy in the context of an active-controlled trial, which does not include a placebo arm. In this report, we provide a summary of the discussion points that included gaps in understanding and logical next steps in the prevention research pathway. The technical challenges involved in the statistical design approaches are described in a companion article.

Risk Factors Associated with HIV Acquisition in Males Participating in HIV Vaccine Efficacy Trials in South Africa

In South Africa, HIV acquisition risk has been studied less in people assigned male at birth. We studied the associations between risk behaviors, clinical features and HIV incidence amongst males in two South African HIV preventive vaccine efficacy trials. We used Cox proportional hazards models to test for associations between demographics, sexual behaviors, clinical variables and HIV acquisition among males followed in the HVTN 503 (n = 219) and HVTN 702 (n = 1611) trials. Most males reported no male sexual partners (99.09% in HVTN 503) or identified as heterosexual (88.08% in HVTN 702). Annual HIV incidence was 1.39% in HVTN 503 (95% CI 0.76-2.32%) and 1.33% in HVTN 702 (95% CI 0.80-2.07%). Increased HIV acquisition was significantly associated with anal sex (HR 6.32, 95% CI 3.44-11.62), transactional sex (HR 3.42, 95% CI 1.80-6.50), and non-heterosexual identity (HR 16.23, 95%CI 8.13-32.41) in univariate analyses and non-heterosexual identity (HR 14.99, 95% CI 4.99-45.04; p < 0.01) in multivariate analysis. It is appropriate that prevention efforts in South Africa, although focused on the severe epidemic in young women, also encompass key male populations, including men who have sex with men, but also men who engage in anal or transactional sex.

Clinical and Demographic Factors Associated With COVID-19, Severe COVID-19, and SARS-CoV-2 Infection in Adults: A Secondary Cross-Protocol Analysis of 4 Randomized Clinical Trials

Importance

Current data identifying COVID-19 risk factors lack standardized outcomes and insufficiently control for confounders.

Objective

To identify risk factors associated with COVID-19, severe COVID-19, and SARS-CoV-2 infection.

Design, Setting, and Participants

This secondary cross-protocol analysis included 4 multicenter, international, randomized, blinded, placebo-controlled, COVID-19 vaccine efficacy trials with harmonized protocols established by the COVID-19 Prevention Network. Individual-level data from participants randomized to receive placebo within each trial were combined and analyzed. Enrollment began July 2020 and the last data cutoff was in July 2021. Participants included adults in stable health, at risk for SARS-CoV-2, and assigned to the placebo group within each vaccine trial. Data were analyzed from April 2022 to February 2023.

Exposures

Comorbid conditions, demographic factors, and SARS-CoV-2 exposure risk at the time of enrollment.

Main Outcomes and Measures

Coprimary outcomes were COVID-19 and severe COVID-19. Multivariate Cox proportional regression models estimated adjusted hazard ratios (aHRs) and 95% CIs for baseline covariates, accounting for trial, region, and calendar time. Secondary outcomes included severe COVID-19 among people with COVID-19, subclinical SARS-CoV-2 infection, and SARS-CoV-2 infection.

Results

A total of 57 692 participants (median [range] age, 51 [18-95] years; 11 720 participants [20.3%] aged ≥65 years; 31 058 participants [53.8%] assigned male at birth) were included. The analysis population included 3270 American Indian or Alaska Native participants (5.7%), 7849 Black or African American participants (13.6%), 17 678 Hispanic or Latino participants (30.6%), and 40 745 White participants (70.6%). Annualized incidence was 13.9% (95% CI, 13.3%-14.4%) for COVID-19 and 2.0% (95% CI, 1.8%-2.2%) for severe COVID-19. Factors associated with increased rates of COVID-19 included workplace exposure (high vs low: aHR, 1.35 [95% CI, 1.16-1.58]; medium vs low: aHR, 1.41 [95% CI, 1.21-1.65]; P < .001) and living condition risk (very high vs low risk: aHR, 1.41 [95% CI, 1.21-1.66]; medium vs low risk: aHR, 1.19 [95% CI, 1.08-1.32]; P < .001). Factors associated with decreased rates of COVID-19 included previous SARS-CoV-2 infection (aHR, 0.13 [95% CI, 0.09-0.19]; P < .001), age 65 years or older (aHR vs age <65 years, 0.57 [95% CI, 0.50-0.64]; P < .001) and Black or African American race (aHR vs White race, 0.78 [95% CI, 0.67-0.91]; P = .002). Factors associated with increased rates of severe COVID-19 included race (American Indian or Alaska Native vs White: aHR, 2.61 [95% CI, 1.85-3.69]; multiracial vs White: aHR, 2.19 [95% CI, 1.50-3.20]; P < .001), diabetes (aHR, 1.54 [95% CI, 1.14-2.08]; P = .005) and at least 2 comorbidities (aHR vs none, 1.39 [95% CI, 1.09-1.76]; P = .008). In analyses restricted to participants who contracted COVID-19, increased severe COVID-19 rates were associated with age 65 years or older (aHR vs <65 years, 1.75 [95% CI, 1.32-2.31]; P < .001), race (American Indian or Alaska Native vs White: aHR, 1.98 [95% CI, 1.38-2.83]; Black or African American vs White: aHR, 1.49 [95% CI, 1.03-2.14]; multiracial: aHR, 1.81 [95% CI, 1.21-2.69]; overall P = .001), body mass index (aHR per 1-unit increase, 1.03 [95% CI, 1.01-1.04]; P = .001), and diabetes (aHR, 1.85 [95% CI, 1.37-2.49]; P < .001). Previous SARS-CoV-2 infection was associated with decreased severe COVID-19 rates (aHR, 0.04 [95% CI, 0.01-0.14]; P < .001).

Conclusions and Relevance

In this secondary cross-protocol analysis of 4 randomized clinical trials, exposure and demographic factors had the strongest associations with outcomes; results could inform mitigation strategies for SARS-CoV-2 and viruses with comparable epidemiological characteristics.

Examining protective effects of SARS-CoV-2 neutralizing antibodies after vaccination or monoclonal antibody administration

While new vaccines for SARS-CoV-2 are authorized based on neutralizing antibody (nAb) titer against emerging variants of concern, an analogous pathway does not exist for preventative monoclonal antibodies. In this work, nAb titers were assessed as correlates of protection against COVID-19 in the casirivimab + imdevimab monoclonal antibody (mAb) prevention trial (ClinicalTrials.gov #NCT4452318) and in the mRNA-1273 vaccine trial (ClinicalTrials.gov #NCT04470427). In the mAb trial, protective efficacy of 92% (95% confidence interval (CI): 84%, 98%) is associated with a nAb titer of 1000 IU50/ml, with lower efficacy at lower nAb titers. In the vaccine trial, protective efficacies of 93% [95% CI: 91%, 95%] and 97% (95% CI: 95%, 98%) are associated with nAb titers of 100 and 1000 IU50/ml, respectively. These data quantitate a nAb titer correlate of protection for mAbs benchmarked alongside vaccine induced nAb titers and support nAb titer as a surrogate endpoint for authorizing new mAbs.

Rapid Development of an Integrated Network Infrastructure to Conduct Phase 3 COVID-19 Vaccine Trials

Importance

The COVID-19 pandemic has caused millions of infections and deaths and resulted in unprecedented international public health social and economic crises. As SARS-CoV-2 spread across the globe and its impact became evident, the development of safe and effective vaccines became a priority. Outlining the processes used to establish and support the conduct of the phase 3 randomized clinical trials that led to the rapid emergency use authorization and approval of several COVID-19 vaccines is of major significance for current and future pandemic response efforts.

Observations

To support the rapid development of vaccines for the US population and the rest of the world, the National Institute of Allergy and Infectious Diseases established the COVID-19 Prevention Network (CoVPN) to assist in the coordination and implementation of phase 3 efficacy trials for COVID-19 vaccine candidates and monoclonal antibodies. By bringing together multiple networks, CoVPN was able to draw on existing clinical and laboratory infrastructure, community partnerships, and research expertise to quickly pivot clinical trial sites to conduct COVID-19 vaccine trials as soon as the investigational products were ready for phase 3 testing. The mission of CoVPN was to operationalize phase 3 vaccine trials using harmonized protocols, laboratory assays, and a single data and safety monitoring board to oversee the various studies. These trials, while staggered in time of initiation, overlapped in time and course of conduct and ultimately led to the successful completion of multiple studies and US Food and Drug Administration-licensed or -authorized vaccines, the first of which was available to the public less than 1 year from the discovery of the virus.

Conclusions and Relevance

This Special Communication describes the design, geographic distribution, and underlying principles of conduct of these efficacy trials and summarizes data from 136 382 prospectively followed-up participants, including more than 2500 with documented COVID-19. These successful efforts can be replicated for other important research initiatives and point to the importance of investments in clinical trial infrastructure integral to pandemic preparedness.

Durability of protection and immunogenicity of AZD1222 (ChAdOx1 nCoV-19) COVID-19 vaccine over 6 months

BackgroundWe report updated safety, efficacy, and immunogenicity of AZD1222 (ChAdOx1 nCoV-19) from an ongoing phase 3 trial.MethodsAdults at increased risk of SARS-CoV-2 infection were randomized (2:1), stratified by age, to receive 2 doses of AZD1222 or placebo. The primary efficacy end point was confirmed SARS-CoV-2 reverse-transcriptase PCR-positive (RT-PCR-positive) symptomatic COVID-19 at 15 or more days after a second dose in baseline SARS-CoV-2-seronegative participants. The 21,634 and 10,816 participants were randomized to AZD1222 and placebo, respectively.FindingsData cutoff for this analysis was July 30, 2021; median follow-up from second dose was 78 and 71 days for the double-blind period (censoring at unblinding or nonstudy COVID-19 vaccination) and 201 and 82 days for the period to nonstudy COVID-19 vaccination (regardless of unblinding) in the AZD1222 and placebo groups, respectively. For the primary efficacy end point in the double-blind period (141 and 184 events; incidence rates: 39.2 and 118.8 per 1,000 person years), vaccine efficacy was 67.0% (P < 0.001). In the period to nonstudy COVID-19 vaccination, incidence of events remained consistently low and stable through 6 months in the AZD1222 group; for the primary efficacy end point (328 and 219 events; incidence rates: 36.4, 108.4) and severe/critical disease (5 and 13 events; incidence rates: 0.6, 6.4), respective vaccine efficacy estimates were 65.1% and 92.1%. AZD1222 elicited humoral immune responses over time, with waning at day 180. No emergent safety issues were seen.ConclusionAZD1222 is safe and well tolerated, demonstrating durable protection and immunogenicity with median follow-up (AZD1222 group) of 6 months.Trial registrationClinicalTrials.gov NCT04516746.FundingAstraZeneca; US government.

Analysis of the HIV Vaccine Trials Network 702 Phase 2b-3 HIV-1 Vaccine Trial in South Africa Assessing RV144 Antibody and T-Cell Correlates of HIV-1 Acquisition Risk

BACKGROUND

The ALVAC/gp120 + MF59 vaccines in the HIV Vaccine Trials Network (HVTN) 702 efficacy trial did not prevent human immunodeficiency virus-1 (HIV-1) acquisition. Vaccine-matched immunological endpoints that were correlates of HIV-1 acquisition risk in RV144 were measured in HVTN 702 and evaluated as correlates of HIV-1 acquisition.

METHODS

Among 1893 HVTN 702 female vaccinees, 60 HIV-1-seropositive cases and 60 matched seronegative noncases were sampled. HIV-specific CD4+ T-cell and binding antibody responses were measured 2 weeks after fourth and fifth immunizations. Cox proportional hazards models assessed prespecified responses as predictors of HIV-1 acquisition.

RESULTS

The HVTN 702 Env-specific CD4+ T-cell response rate was significantly higher than in RV144 (63% vs 40%, P = .03) with significantly lower IgG binding antibody response rate and magnitude to 1086.C V1V2 (67% vs 100%, P < .001; Pmag < .001). Although no significant univariate associations were observed between any T-cell or binding antibody response and HIV-1 acquisition, significant interactions were observed (multiplicity-adjusted P ≤.03). Among vaccinees with high IgG A244 V1V2 binding antibody responses, vaccine-matched CD4+ T-cell endpoints associated with decreased HIV-1 acquisition (estimated hazard ratios = 0.40-0.49 per 1-SD increase in CD4+ T-cell endpoint).

CONCLUSIONS

HVTN 702 and RV144 had distinct immunogenicity profiles. However, both identified significant correlations (univariate or interaction) for IgG V1V2 and polyfunctional CD4+ T cells with HIV-1 acquisition. Clinical Trials Registration . NCT02968849.

Could widespread use of antiviral treatment curb the COVID-19 pandemic? A modeling study

BACKGROUND

Despite the development of safe and effective vaccines, effective treatments for COVID-19 disease are still urgently needed. Several antiviral drugs have shown to be effective in reducing progression of COVID-19 disease.

METHODS

In the present work, we use an agent-based mathematical model to assess the potential population impact of the use of antiviral treatments in four countries with different demographic structure and current levels of vaccination coverage: Kenya, Mexico, United States (US) and Belgium. We analyzed antiviral effects on reducing hospitalization and death, and potential antiviral effects on reducing transmission. For each country, we varied daily treatment initiation rate (DTIR) and antiviral effect in reducing transmission (AVT).

RESULTS

Irrespective of location and AVT, widespread antiviral treatment of symptomatic adult infections (20% DTIR) prevented the majority of COVID-19 deaths, and recruiting 6% of all adult symptomatic infections daily reduced mortality by over 20% in all countries. Furthermore, our model projected that targeting antiviral treatment to the oldest age group (65 years old and older, DTIR of 20%) can prevent over 30% of deaths. Our results suggest that early antiviral treatment (as soon as possible after inception of infection) is needed to mitigate transmission, preventing 50% more infections compared to late treatment (started 3 to 5 days after symptoms onset). Our results highlight the synergistic effect of vaccination and antiviral treatment: as the vaccination rate increases, antivirals have a larger relative impact on population transmission. Finally, our model projects that even in highly vaccinated populations, adding antiviral treatment can be extremely helpful to mitigate COVID-19 deaths.

CONCLUSIONS

These results suggest that antiviral treatments can become a strategic tool that, in combination with vaccination, can significantly reduce COVID-19 hospitalizations and deaths and can help control SARS-CoV-2 transmission.

Mycobacterium tuberculosis infection, immune activation, and risk of HIV acquisition

BACKGROUND

Although immune activation is associated with HIV acquisition, the nature of inflammatory profiles that increase HIV risk, which may include responses to M. tuberculosis (Mtb) infection, are not well characterized.

METHODS

We conducted a nested case-control study using cryopreserved samples from persons who did and did not acquire HIV during the multinational Step clinical trial of the MRKAd5 HIV-1 vaccine. PBMCs from the last HIV-negative sample from incident HIV cases and controls were stimulated with Mtb-specific antigens (ESAT-6/CFP-10) and analyzed by flow cytometry with intracellular cytokine staining and scored with COMPASS. We measured inflammatory profiles with five Correlates of TB Risk (CoR) transcriptomic signatures. Our primary analysis examined the association of latent Mtb infection (LTBI; IFNγ+CD4+ T cell frequency) or RISK6 CoR signature with HIV acquisition. Conditional logistic regression analyses, adjusted for known predictors of HIV acquisition, were employed to assess whether TB-associated immune markers were associated with HIV acquisition.

RESULTS

Among 465 participants, LTBI prevalence (21.5% controls vs 19.1% cases, p = 0.51) and the RISK6 signature were not higher in those who acquired HIV. In exploratory analyses, Mtb antigen-specific polyfunctional CD4+ T cell COMPASS scores (aOR 0.96, 95% CI 0.77, 1.20) were not higher in those who acquired HIV. Two CoR signatures, Sweeney3 (aOR 1.38 (1.07, 1.78) per SD change) and RESPONSE5 (0.78 (0.61, 0.98)), were associated with HIV acquisition. The transcriptomic pattern used to differentiate active vs latent TB (Sweeney3) was most strongly associated with acquiring HIV.

CONCLUSIONS

LTBI, Mtb polyfunctional antigen-specific CD4+ T cell activation, and RISK6 were not identified as risks for HIV acquisition. In exploratory transcriptomic analyses, two CoR signatures were associated with HIV risk after adjustment for known behavioral and clinical risk factors. We identified host gene expression signatures associated with HIV acquisition, but the observed effects are likely not mediated through Mtb infection.

Could widespread use of antiviral treatment curb the COVID-19 pandemic? A modeling study

Despite the development of safe and effective vaccines, effective treatments for COVID-19 disease are still desperately needed. Recently, two antiviral drugs have shown to be effective in reducing hospitalizations in clinical trials. In the present work, we use an agent-based mathematical model to assess the potential population impact of the use of antiviral treatments in four countries, corresponding to four current levels of vaccination coverage: Kenya, Mexico, United States (US) and Belgium, with 1.5, 38, 57 and 74% of their populations vaccinated. For each location, we varied antiviral coverage and antiviral effect in reducing viral load (25, 50, 75 or 100% reduction). Irrespective of location, widespread antiviral treatment of symptomatic infections (≥50% coverage) is expected to prevent the majority of COVID-19 deaths. Furthermore, even treating 20% of adult symptomatic infections, is expected to reduce mortality by a third in all countries, irrespective of the assumed treatment efficacy in reducing viral load. Our results suggest that early antiviral treatment is needed to mitigate transmission, with early treatment (within two days of symptoms) preventing 50% more infections compared to late treatment (started on days 3 to 5 after developing symptoms). Our results highlight the synergistic effect of vaccination and antiviral treatment: as vaccination rate increased, antiviral treatment had a bigger impact on overall transmission. These results suggest that antiviral treatments can become a strategic tool that, in combination with vaccination, can significantly control SASRS-CoV-2 transmission and reduce COVID-19 hospitalizations and deaths.

COVID-19 Vaccines and SARS-CoV-2 Transmission in the Era of New Variants: A Review and Perspective

Coronavirus disease 2019 (COVID-19) vaccines have yielded definitive prevention and major reductions in morbidity and mortality from severe acute respiratory syndrome coronavirus 2 infection, even in the context of emerging and persistent variants of concern. Newer variants have revealed less vaccine protection against infection and attenuation of vaccine effects on transmission. COVID-19 vaccines still likely reduce transmission compared with not being vaccinated at all, even with variants of concern; however, determining the magnitude of transmission reduction is constrained by the challenges of performing these studies, requiring accurate linkage of infections to vaccine status and timing thereof, particularly within households. In this review, we synthesize the currently available data on the impact of COVID-19 vaccines on infection, serious illness, and transmission; we also identify the challenges and opportunities associated with policy development based on this data.

Efficacy of the mRNA-1273 SARS-CoV-2 Vaccine at Completion of Blinded Phase

BACKGROUND

At interim analysis in a phase 3, observer-blinded, placebo-controlled clinical trial, the mRNA-1273 vaccine showed 94.1% efficacy in preventing coronavirus disease 2019 (Covid-19). After emergency use of the vaccine was authorized, the protocol was amended to include an open-label phase. Final analyses of efficacy and safety data from the blinded phase of the trial are reported.

METHODS

We enrolled volunteers who were at high risk for Covid-19 or its complications; participants were randomly assigned in a 1:1 ratio to receive two intramuscular injections of mRNA-1273 (100 μg) or placebo, 28 days apart, at 99 centers across the United States. The primary end point was prevention of Covid-19 illness with onset at least 14 days after the second injection in participants who had not previously been infected with the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). The data cutoff date was March 26, 2021.

RESULTS

The trial enrolled 30,415 participants; 15,209 were assigned to receive the mRNA-1273 vaccine, and 15,206 to receive placebo. More than 96% of participants received both injections, 2.3% had evidence of SARS-CoV-2 infection at baseline, and the median follow-up was 5.3 months in the blinded phase. Vaccine efficacy in preventing Covid-19 illness was 93.2% (95% confidence interval [CI], 91.0 to 94.8), with 55 confirmed cases in the mRNA-1273 group (9.6 per 1000 person-years; 95% CI, 7.2 to 12.5) and 744 in the placebo group (136.6 per 1000 person-years; 95% CI, 127.0 to 146.8). The efficacy in preventing severe disease was 98.2% (95% CI, 92.8 to 99.6), with 2 cases in the mRNA-1273 group and 106 in the placebo group, and the efficacy in preventing asymptomatic infection starting 14 days after the second injection was 63.0% (95% CI, 56.6 to 68.5), with 214 cases in the mRNA-1273 group and 498 in the placebo group. Vaccine efficacy was consistent across ethnic and racial groups, age groups, and participants with coexisting conditions. No safety concerns were identified.

CONCLUSIONS

The mRNA-1273 vaccine continued to be efficacious in preventing Covid-19 illness and severe disease at more than 5 months, with an acceptable safety profile, and protection against asymptomatic infection was observed. (Funded by the Biomedical Advanced Research and Development Authority and the National Institute of Allergy and Infectious Diseases; COVE ClinicalTrials.gov number, NCT04470427.).

Mathematical Modeling of Vaccines That Prevent SARS-CoV-2 Transmission

SARS-CoV-2 vaccine clinical trials assess efficacy against disease (VEDIS), the ability to block symptomatic COVID-19. They only partially discriminate whether VEDIS is mediated by preventing infection completely, which is defined as detection of virus in the airways (VESUSC), or by preventing symptoms despite infection (VESYMP). Vaccine efficacy against transmissibility given infection (VEINF), the decrease in secondary transmissions from infected vaccine recipients, is also not measured. Using mathematical modeling of data from King County Washington, we demonstrate that if the Moderna (mRNA-1273QS) and Pfizer-BioNTech (BNT162b2) vaccines, which demonstrated VEDIS > 90% in clinical trials, mediate VEDIS by VESUSC, then a limited fourth epidemic wave of infections with the highly infectious B.1.1.7 variant would have been predicted in spring 2021 assuming rapid vaccine roll out. If high VEDIS is explained by VESYMP, then high VEINF would have also been necessary to limit the extent of this fourth wave. Vaccines which completely protect against infection or secondary transmission also substantially lower the number of people who must be vaccinated before the herd immunity threshold is reached. The limited extent of the fourth wave suggests that the vaccines have either high VESUSC or both high VESYMP and high VEINF against B.1.1.7. Finally, using a separate intra-host mathematical model of viral kinetics, we demonstrate that a 0.6 log vaccine-mediated reduction in average peak viral load might be sufficient to achieve 50% VEINF, which suggests that human challenge studies with a relatively low number of infected participants could be employed to estimate all three vaccine efficacy metrics.

RV144 vaccine imprinting constrained HIV-1 evolution following breakthrough infection

The scale of the HIV-1 epidemic underscores the need for a vaccine. The multitude of circulating HIV-1 strains together with HIV-1’s high evolvability hints that HIV-1 could adapt to a future vaccine. Here, we wanted to investigate the effect of vaccination on the evolution of the virus post-breakthrough infection. We analyzed 2,635 HIV-1 env sequences sampled up to a year post-diagnosis from 110 vaccine and placebo participants who became infected in the RV144 vaccine efficacy trial. We showed that the Env signature sites that were previously identified to distinguish vaccine and placebo participants were maintained over time. In addition, fewer sites were under diversifying selection in the vaccine group than in the placebo group. These results indicate that HIV-1 would possibly adapt to a vaccine upon its roll-out.

COVID-19 vaccines that reduce symptoms but do not block infection need higher coverage and faster rollout to achieve population impact

Trial results for two COVID-19 vaccines suggest at least 90% efficacy against symptomatic disease (VEDIS). It remains unknown whether this efficacy is mediated by lowering SARS-CoV-2 infection susceptibility (VESUSC) or development of symptoms after infection (VESYMP). We aim to assess and compare the population impact of vaccines with different efficacy profiles (VESYMP and VESUSC) satisfying licensure criteria. We developed a mathematical model of SARS-CoV-2 transmission, calibrated to data from King County, Washington. Rollout scenarios starting December 2020 were simulated with combinations of VESUSC and VESYMP resulting in up to 100% VEDIS. We assumed no reduction of infectivity upon infection conditional on presence of symptoms. Proportions of cumulative infections, hospitalizations and deaths prevented over 1 year from vaccination start are reported. Rollouts of 1 M vaccinations (5000 daily) using vaccines with 50% VEDIS are projected to prevent 23-46% of infections and 31-46% of deaths over 1 year. In comparison, vaccines with 90% VEDIS are projected to prevent 37-64% of infections and 46-64% of deaths over 1 year. In both cases, there is a greater reduction if VEDIS is mediated mostly by VESUSC. The use of a "symptom reducing" vaccine will require twice as many people vaccinated than a "susceptibility reducing" vaccine with the same 90% VEDIS to prevent 50% of the infections and death over 1 year. Delaying the start of the vaccination by 3 months decreases the expected population impact by more than 50%. Vaccines which prevent COVID-19 disease but not SARS-CoV-2 infection, and thereby shift symptomatic infections to asymptomatic infections, will prevent fewer infections and require larger and faster vaccination rollouts to have population impact, compared to vaccines that reduce susceptibility to infection. If uncontrolled transmission across the U.S. continues, then expected vaccination in Spring 2021 will provide only limited benefit.

Quantifying the Impact of Lifting Community Nonpharmaceutical Interventions for COVID-19 During Vaccination Rollout in the United States

Using a mathematical model, we estimated the potential impact on mortality and total infections of completely lifting community nonpharmaceutical interventions when only a small proportion of the population has been fully vaccinated in 2 states in the United States. Lifting all community nonpharmaceutical interventions immediately is predicted to result in twice as many deaths over the next 6 months as a more moderate reopening allowing 70% of prepandemic contacts.

Optimizing vaccine allocation for COVID-19 vaccines shows the potential role of single-dose vaccination

Most COVID-19 vaccines require two doses, however with limited vaccine supply, policymakers are considering single-dose vaccination as an alternative strategy. Using a mathematical model combined with optimization algorithms, we determined optimal allocation strategies with one and two doses of vaccine under various degrees of viral transmission. Under low transmission, we show that the optimal allocation of vaccine vitally depends on the single-dose efficacy. With high single-dose efficacy, single-dose vaccination is optimal, preventing up to 22% more deaths than a strategy prioritizing two-dose vaccination for older adults. With low or moderate single-dose efficacy, mixed vaccination campaigns with complete coverage of older adults are optimal. However, with modest or high transmission, vaccinating older adults first with two doses is best, preventing up to 41% more deaths than a single-dose vaccination given across all adult populations. Our work suggests that it is imperative to determine the efficacy and durability of single-dose vaccines, as mixed or single-dose vaccination campaigns may have the potential to contain the pandemic much more quickly.

Optimizing vaccine allocation for COVID-19 vaccines: potential role of single-dose vaccination

Most COVID-19 vaccines require two doses, however with limited vaccine supply, policymakers are considering single-dose vaccination as an alternative strategy. Using a mathematical model combined with optimization algorithms, we determined optimal allocation strategies with one and two doses of vaccine under various degrees of viral transmission. Under low transmission, we show that the optimal allocation of vaccine vitally depends on the single-dose efficacy (SDE). With high SDE, single-dose vaccination is optimal, preventing up to 22% more deaths than a strategy prioritizing two-dose vaccination for older adults. With low or moderate SDE, mixed vaccination campaigns with complete coverage of older adults are optimal. However, with modest or high transmission, vaccinating older adults first with two doses is best, preventing up to 41% more deaths than a single-dose vaccination given across all adult populations. Our work suggests that it is imperative to determine the efficacy and durability of single-dose vaccines, as mixed or single-dose vaccination campaigns may have the potential to contain the pandemic much more quickly.

Vaccine Efficacy of ALVAC-HIV and Bivalent Subtype C gp120-MF59 in Adults

BACKGROUND

A safe, effective vaccine is essential to eradicating human immunodeficiency virus (HIV) infection. A canarypox-protein HIV vaccine regimen (ALVAC-HIV plus AIDSVAX B/E) showed modest efficacy in reducing infection in Thailand. An analogous regimen using HIV-1 subtype C virus showed potent humoral and cellular responses in a phase 1-2a trial in South Africa. Efficacy data and additional safety data were needed for this regimen in a larger population in South Africa.

METHODS

In this phase 2b-3 trial, we randomly assigned 5404 adults without HIV-1 infection to receive the vaccine (2704 participants) or placebo (2700 participants). The vaccine regimen consisted of injections of ALVAC-HIV at months 0 and 1, followed by four booster injections of ALVAC-HIV plus bivalent subtype C gp120-MF59 adjuvant at months 3, 6, 12, and 18. The primary efficacy outcome was the occurrence of HIV-1 infection from randomization to 24 months.

RESULTS

In January 2020, prespecified criteria for nonefficacy were met at an interim analysis; further vaccinations were subsequently halted. The median age of the trial participants was 24 years; 70% of the participants were women. The incidence of adverse events was similar in the vaccine and placebo groups. During the 24-month follow-up, HIV-1 infection was diagnosed in 138 participants in the vaccine group and in 133 in the placebo group (hazard ratio, 1.02; 95% confidence interval, 0.81 to 1.30; P = 0.84).

CONCLUSIONS

The ALVAC-gp120 regimen did not prevent HIV-1 infection among participants in South Africa despite previous evidence of immunogenicity. (HVTN 702 ClinicalTrials.gov number, NCT02968849.).

Efficacy and Safety of the mRNA-1273 SARS-CoV-2 Vaccine

BACKGROUND

Vaccines are needed to prevent coronavirus disease 2019 (Covid-19) and to protect persons who are at high risk for complications. The mRNA-1273 vaccine is a lipid nanoparticle-encapsulated mRNA-based vaccine that encodes the prefusion stabilized full-length spike protein of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), the virus that causes Covid-19.

METHODS

This phase 3 randomized, observer-blinded, placebo-controlled trial was conducted at 99 centers across the United States. Persons at high risk for SARS-CoV-2 infection or its complications were randomly assigned in a 1:1 ratio to receive two intramuscular injections of mRNA-1273 (100 μg) or placebo 28 days apart. The primary end point was prevention of Covid-19 illness with onset at least 14 days after the second injection in participants who had not previously been infected with SARS-CoV-2.

RESULTS

The trial enrolled 30,420 volunteers who were randomly assigned in a 1:1 ratio to receive either vaccine or placebo (15,210 participants in each group). More than 96% of participants received both injections, and 2.2% had evidence (serologic, virologic, or both) of SARS-CoV-2 infection at baseline. Symptomatic Covid-19 illness was confirmed in 185 participants in the placebo group (56.5 per 1000 person-years; 95% confidence interval [CI], 48.7 to 65.3) and in 11 participants in the mRNA-1273 group (3.3 per 1000 person-years; 95% CI, 1.7 to 6.0); vaccine efficacy was 94.1% (95% CI, 89.3 to 96.8%; P<0.001). Efficacy was similar across key secondary analyses, including assessment 14 days after the first dose, analyses that included participants who had evidence of SARS-CoV-2 infection at baseline, and analyses in participants 65 years of age or older. Severe Covid-19 occurred in 30 participants, with one fatality; all 30 were in the placebo group. Moderate, transient reactogenicity after vaccination occurred more frequently in the mRNA-1273 group. Serious adverse events were rare, and the incidence was similar in the two groups.

CONCLUSIONS

The mRNA-1273 vaccine showed 94.1% efficacy at preventing Covid-19 illness, including severe disease. Aside from transient local and systemic reactions, no safety concerns were identified. (Funded by the Biomedical Advanced Research and Development Authority and the National Institute of Allergy and Infectious Diseases; COVE ClinicalTrials.gov number, NCT04470427.).

Efficacy and Safety of the mRNA-1273 SARS-CoV-2 Vaccine

BACKGROUND

Vaccines are needed to prevent coronavirus disease 2019 (Covid-19) and to protect persons who are at high risk for complications. The mRNA-1273 vaccine is a lipid nanoparticle-encapsulated mRNA-based vaccine that encodes the prefusion stabilized full-length spike protein of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), the virus that causes Covid-19.

METHODS

This phase 3 randomized, observer-blinded, placebo-controlled trial was conducted at 99 centers across the United States. Persons at high risk for SARS-CoV-2 infection or its complications were randomly assigned in a 1:1 ratio to receive two intramuscular injections of mRNA-1273 (100 μg) or placebo 28 days apart. The primary end point was prevention of Covid-19 illness with onset at least 14 days after the second injection in participants who had not previously been infected with SARS-CoV-2.

RESULTS

The trial enrolled 30,420 volunteers who were randomly assigned in a 1:1 ratio to receive either vaccine or placebo (15,210 participants in each group). More than 96% of participants received both injections, and 2.2% had evidence (serologic, virologic, or both) of SARS-CoV-2 infection at baseline. Symptomatic Covid-19 illness was confirmed in 185 participants in the placebo group (56.5 per 1000 person-years; 95% confidence interval [CI], 48.7 to 65.3) and in 11 participants in the mRNA-1273 group (3.3 per 1000 person-years; 95% CI, 1.7 to 6.0); vaccine efficacy was 94.1% (95% CI, 89.3 to 96.8%; P<0.001). Efficacy was similar across key secondary analyses, including assessment 14 days after the first dose, analyses that included participants who had evidence of SARS-CoV-2 infection at baseline, and analyses in participants 65 years of age or older. Severe Covid-19 occurred in 30 participants, with one fatality; all 30 were in the placebo group. Moderate, transient reactogenicity after vaccination occurred more frequently in the mRNA-1273 group. Serious adverse events were rare, and the incidence was similar in the two groups.

CONCLUSIONS

The mRNA-1273 vaccine showed 94.1% efficacy at preventing Covid-19 illness, including severe disease. Aside from transient local and systemic reactions, no safety concerns were identified. (Funded by the Biomedical Advanced Research and Development Authority and the National Institute of Allergy and Infectious Diseases; COVE ClinicalTrials.gov number, NCT04470427.).

Designing HIV Vaccine Efficacy Trials in the Context of Highly Effective Non-vaccine Prevention Modalities

The evolving HIV prevention landscape poses challenges to the statistical design of future trials of candidate HIV vaccines. Study designs must address the anticipated reduction in HIV incidence due to adding new prevention modalities to the standard prevention package provided to trial participants, and must also accommodate individual choices of participants with regard to the use of these modalities. We explore four potential trial designs that address these challenges, with a focus on accommodating the newest addition to the prevention package-antiretroviral-based oral pre-exposure prophylaxis (PrEP). The designs differ with respect to how individuals who take up oral PrEP at screening are handled. An All-Comers Design enrolls and randomizes all eligible individuals, a Decliners Design enrolls and randomizes only those who decline PrEP at screening, and Single and Multi-Stage Run-In Designs enroll all but randomize only those who decline PrEP or show inadequate adherence to PrEP after one or multiple run-in periods. We compare these designs with respect to required sample sizes, study duration, and resource requirements, using a simulation model that incorporates data on HIV risk and PrEP uptake and adherence among men who have sex with men (MSM) in the Americas. We advocate considering Run-In Designs for some future contexts, and identify their advantages and tradeoffs relative to the other designs. The design concepts apply beyond HIV vaccines to other prevention modalities being developed with the aim to achieve further reductions in HIV incidence.

The association of α4β7 expression with HIV acquisition and disease progression in people who inject drugs and men who have sex with men: Case control studies

BACKGROUND

α4β7 is a gut-homing integrin heterodimer that can act as a non-essential binding molecule for HIV. A previous study in heterosexual African women found that individuals with higher proportions of α4β7 expressing CD4⁺ T cells were more likely to become infected with HIV, as well as present with faster disease progression. It is unknown if this phenomenon is also observed in men who have sex with men (MSM) or people who inject drugs (PWID).

METHODS

MSM and transgender women who seroconverted as part of the HVTN 505 HIV vaccine trial and PWID who seroconverted during the ALIVE cohort study were selected as cases and matched to HIV-uninfected controls from the same studies (1:1 and 1:3, respectively). Pre-seroconversion PBMC samples from cases and controls in both studies were examined by flow cytometry to measure levels of α4β7 expression on CD4⁺ T cells. Multivariable conditional logistic regression was used to compare α4β7 expression levels between cases and controls. A Kaplan-Meier curve was used to examine the association of α4β7 expression pre-seroconversion with HIV disease progression.

FINDINGS

In MSM and transgender women (n = 103 cases, 103 controls), there was no statistically significant difference in the levels of α4β7 expression on CD4⁺ T cells between cases and controls (adjusted odds ratio [adjOR] =1.10, 95% confidence interval [CI]=0.94,1.29; p = 0.246). Interestingly, in PWID (n = 49 cases, 143 controls), cases had significantly lower levels of α4β7 expression compared to their matched controls (adjOR = 0.80, 95% CI = 0.68, 0.93; p = 0.004). Among HIV-positive PWID (n = 47), there was no significant association in HIV disease progression in individuals above or below the median level of α4β7 expression (log-rank p = 0.84).

INTERPRETATION

In contrast to findings in heterosexual women, higher α4β7 expression does not predict HIV acquisition or disease progression in PWID or MSM.

FUNDING

This study was supported in part by the Division of Intramural Research, National Institute of Allergy and Infectious Diseases (NIAID), National Institutes of Health. The study was also supported by extramural grants from NIAID T32AI102623 (E.U.P.), and UM1AI069470.

Statistical inference for net benefit measures in biomarker validation studies

Referral strategies based on risk scores and medical tests are commonly proposed. Direct assessment of their clinical utility requires implementing the strategy and is not possible in the early phases of biomarker research. Prior to late-phase studies, net benefit measures can be used to assess the potential clinical impact of a proposed strategy. Validation studies, in which the biomarker defines a prespecified referral strategy, are a gold standard approach to evaluating biomarker potential. Uncertainty, quantified by a confidence interval, is important to consider when deciding whether a biomarker warrants an impact study, does not demonstrate clinical potential, or that more data are needed. We establish distribution theory for empirical estimators of net benefit and propose empirical estimators of variance. The primary results are for the most commonly employed estimators of net benefit: from cohort and unmatched case-control samples, and for point estimates and net benefit curves. Novel estimators of net benefit under stratified two-phase and categorically matched case-control sampling are proposed and distribution theory developed. Results for common variants of net benefit and for estimation from right-censored outcomes are also presented. We motivate and demonstrate the methodology with examples from lung cancer research and highlight its application to study design.

Impact of vaccine type on HIV-1 vaccine elicited antibody durability and B cell gene signature

Efficacious HIV-1 vaccination requires elicitation of long-lived antibody responses. However, our understanding of how different vaccine types elicit durable antibody responses is lacking. To assess the impact of vaccine type on antibody responses, we measured IgG isotypes against four consensus HIV antigens from 2 weeks to 10 years post HIV-1 vaccination and used mixed effects models to estimate half-life of responses in four human clinical trials. Compared to protein-boosted regimens, half-lives of gp120-specific antibodies were longer but peak magnitudes were lower in Modified Vaccinia Ankara (MVA)-boosted regimens. Furthermore, gp120-specific B cell transcriptomics from MVA-boosted and protein-boosted vaccines revealed a distinct signature at a peak (2 weeks after last vaccination) including CD19, CD40, and FCRL2-5 activation along with increased B cell receptor signaling. Additional analysis revealed contributions of RIG-I-like receptor pathway and genes such as SMAD5 and IL-32 to antibody durability. Thus, this study provides novel insights into vaccine induced antibody durability and B-cell receptor signaling.

Safety and immunogenicity of a multivalent HIV vaccine comprising envelope protein with either DNA or NYVAC vectors (HVTN 096): a phase 1b, double-blind, placebo-controlled trial

BACKGROUND

Up to now, immunisation regimens that have been assessed for development of HIV vaccines have included purified envelope (Env) protein among the boosting components of the regimen. We postulated that co-administration of Env protein with either a DNA or NYVAC vector during priming would result in early generation of antibody responses to the Env V1/V2 region, which are important markers for effective protection against infection. We aimed to assess the safety and immunogenicity of a multivalent HIV vaccine including either DNA or NYVAC vectors alone or in combination with Env glycoprotein (gp120) followed by a co-delivered NYVAC and Env protein boost.

METHODS

We did a single-centre, double-blind, placebo-controlled phase 1b trial at the Centre Hospitalier Universitaire Vaudois (Lausanne, Switzerland). We included healthy volunteers aged 18-50 years who were at low risk of HIV infection. We randomly allocated participants using computer-generated random numbers to one of four vaccination schedules or placebo (4:1), and within these schedules participants were allocated either active treatment (T1, T2, T3, and T4) or placebo (C1, C2, C3, and C4). T1 consisted of two doses of NYVAC vector followed by two doses of NYVAC vector and gp120 Env protein; T2 comprised four doses of NYVAC vector and gp120 Env protein; T3 was two doses of DNA vector followed by two doses of NYVAC vector and gp120 Env protein; and T4 was two doses of DNA vector and gp120 Env protein followed by two doses of NYVAC vector and gp120 Env protein. Placebo injections were matched to the corresponding active treatment group. Doses were administered by injection at months 0, 1, 3, and 6. Primary outcomes were safety and immunogenicity of the vaccine schedules. Immune response measures included cross-clade and epitope-specific binding antibodies, neutralising antibodies, and antibody-dependent cell-mediated cytotoxicity measured 2 weeks after the month 1, 3, and 6 vaccinations. This trial is registered with ClinicalTrials.gov, NCT01799954.

FINDINGS

Between Aug 23, 2012, and April 18, 2013, 148 healthy adult volunteers were screened for the trial, of whom 96 participants were enrolled. 20 individuals were allocated to each active treatment group (groups T1-4; n=80) and four were assigned to each placebo group (groups C1-4; n=16). Vaccines containing the NYVAC vector (groups T1 and T2) were associated with more frequent severe reactogenicity and more adverse events than were vaccines containing the DNA vector (groups T3 and T4). The most frequent adverse events judged related to study product were lymphadenopathy (n=9) and hypoaesthesia (n=2). Two participants, one in the placebo group and one in the DNA-primed T3 group, had serious adverse events that were judged unrelated to study product. One participant in the T3 group died from cranial trauma after a motor vehicle accident. Across the active treatment groups, IgG responses 2 weeks after the 6-month dose of vaccine were 74-95%. Early administration of gp120 Env protein (groups T2 and T4) was associated with a substantially earlier and higher area under the curve for gp120 Env binding, production of anti-V1/V2 and neutralising antibodies, and better antibody-response coverage over a period of 18 months, compared with vaccination regimens that delayed administration of gp120 Env protein until the 3-month vaccination (groups T1 and T3).

INTERPRETATION

Co-administration of gp120 Env protein components with DNA or NYVAC vectors during priming led to early and potent induction of Env V1/V2 IgG binding antibody responses. This immunisation approach should be considered for induction of preventive antibodies in future HIV vaccine efficacy trials.

FUNDING

National Institutes of Health, National Institute of Allergy and Infectious Diseases, and the Bill & Melinda Gates Foundation.

Fc Gamma Receptor Polymorphisms Modulated the Vaccine Effect on HIV-1 Risk in the HVTN 505 HIV Vaccine Trial

HIV Vaccine Trials Network (HVTN) 505 was a phase 2b efficacy trial of a DNA/recombinant adenovirus 5 (rAd5) HIV vaccine regimen. Although the trial was stopped early for lack of overall efficacy, later correlates of risk and sieve analyses generated the hypothesis that the DNA/rAd5 vaccine regimen protected some vaccinees from HIV infection yet enhanced HIV infection risk for others. Here, we assessed whether and how host Fc gamma receptor (FcγR) genetic variations influenced the DNA/rAd5 vaccine regimen's effect on HIV infection risk. We found that vaccine receipt significantly increased HIV acquisition compared with placebo receipt among participants carrying the FCGR2C-TATA haplotype (comprising minor alleles of four FCGR2C single-nucleotide polymorphism [SNP] sites) (hazard ratio [HR] = 9.79, P = 0.035) but not among participants without the haplotype (HR = 0.86, P = 0.67); the interaction of vaccine and haplotype effect was significant (P = 0.034). Similarly, vaccine receipt increased HIV acquisition compared with placebo receipt among participants carrying the FCGR3B-AGA haplotype (comprising minor alleles of the 3 FCGR3B SNPs) (HR = 2.78, P = 0.058) but not among participants without the haplotype (HR = 0.73, P = 0.44); again, the interaction of vaccine and haplotype was significant (P = 0.047). The FCGR3B-AGA haplotype also influenced whether a combined Env-specific CD8+ T-cell polyfunctionality score and IgG response correlated significantly with HIV risk; an FCGR2A SNP and two FCGR2B SNPs influenced whether anti-gp140 antibody-dependent cellular phagocytosis correlated significantly with HIV risk. These results provide further evidence that Fc gamma receptor genetic variations may modulate HIV vaccine effects and immune function after HIV vaccination.IMPORTANCE By analyzing data from the HVTN 505 efficacy trial of a DNA/recombinant adenovirus 5 (rAd5) vaccine regimen, we found that host genetics, specifically Fc gamma receptor genetic variations, influenced whether receiving the DNA/rAd5 regimen was beneficial, neutral, or detrimental to an individual with respect to HIV-1 acquisition risk. Moreover, Fc gamma receptor genetic variations influenced immune responses to the DNA/rAd5 vaccine regimen. Thus, Fc gamma receptor genetic variations should be considered in the analysis of future HIV vaccine trials and the development of HIV vaccines.

Evaluating the impact of policies recommending PrEP to subpopulations of men and transgender women who have sex with men based on demographic and behavioral risk factors

INTRODUCTION

Developing guidelines to inform the use of antiretroviral pre-exposure prophylaxis (PrEP) for HIV prevention in resource-limited settings must necessarily be informed by considering the resources and infrastructure needed for PrEP delivery. We describe an approach that identifies subpopulations of cisgender men who have sex with men (MSM) and transgender women (TGW) to prioritize for the rollout of PrEP in resource-limited settings.

METHODS

We use data from the iPrEx study, a multi-national phase III study of PrEP for HIV prevention in MSM/TGW, to build statistical models that identify subpopulations at high risk of HIV acquisition without PrEP, and with high expected PrEP benefit. We then evaluate empirically the population impact of policies recommending PrEP to these subpopulations, and contrast these with existing policies.

RESULTS

A policy recommending PrEP to a high risk subpopulation of MSM/TGW reporting condomless receptive anal intercourse over the last 3 months (estimated 3.3% 1-year HIV incidence) yields an estimated 1.95% absolute reduction in 1-year HIV incidence at the population level, and 3.83% reduction over 2 years. Importantly, such a policy requires rolling PrEP out to just 59.7% of MSM/TGW in the iPrEx population. We find that this policy is identical to that which prioritizes MSM/TGW with high expected PrEP benefit. It is estimated to achieve nearly the same reduction in HIV incidence as the PrEP guideline put forth by the US Centers for Disease Control, which relies on the measurement of more behavioral risk factors and which would recommend PrEP to a larger subset of the MSM/TGW population (86% vs. 60%).

CONCLUSIONS

These findings may be used to focus future mathematical modelling studies of PrEP in resource-limited settings on prioritizing PrEP for high-risk subpopulations of MSM/TGW. The statistical approach we took could be employed to develop PrEP policies for other at-risk populations and resource-limited settings.

Assessing the Clinical Impact of Risk Models for Opting Out of Treatment

Decision curves are a tool for evaluating the population impact of using a risk model for deciding whether to undergo some intervention, which might be a treatment to help prevent an unwanted clinical event or invasive diagnostic testing such as biopsy. The common formulation of decision curves is based on an opt-in framework. That is, a risk model is evaluated based on the population impact of using the model to opt high-risk patients into treatment in a setting where the standard of care is not to treat. Opt-in decision curves display the population net benefit of the risk model in comparison to the reference policy of treating no patients. In some contexts, however, the standard of care in the absence of a risk model is to treat everyone, and the potential use of the risk model would be to opt low-risk patients out of treatment. Although opt-out settings were discussed in the original decision curve paper, opt-out decision curves are underused. We review the formulation of opt-out decision curves and discuss their advantages for interpretation and inference when treat-all is the standard.

Methods for comparing durability of immune responses between vaccine regimens in early-phase trials

The ability to produce a long-lasting, or durable, immune response is a crucial characteristic of many highly effective vaccines. A goal of early-phase vaccine trials is often to compare the immune response durability of multiple tested vaccine regimens. One parameter for measuring immune response durability is the area under the mean post-peak log immune response profile. In this paper, we compare immune response durability across vaccine regimens within and between two phase I trials of DNA-primed HIV vaccine regimens, HVTN 094 and HVTN 096. We compare four estimators of this durability parameter and the resulting statistical inferences for comparing vaccine regimens. Two of these estimators use the trapezoid rule as an empirical approximation of the area under the marginal log response curve, and the other two estimators are based on linear and nonlinear models for the marginal mean log response. We conduct a simulation study to compare the four estimators, provide guidance on estimator selection, and use the nonlinear marginal mean model to analyze immunogenicity data from the two HIV vaccine trials.

First things first: risk model performance metrics should reflect the clinical application

Developing new measures of risk model performance is an active line of research, often motivated by the conventional wisdom that area under the ROC curve is an 'insensitive' measure of the additional predictive capacity offered by new biomarkers. Without endorsing area under the ROC curve, we argue that this charge is not substantiated. Three articles in this issue discuss alternative metrics of risk model performance: NRI(p) (two-category net reclassification index at the event rate), integrated discrimination index, and R-squared statistics. Guided by the principle that performance metrics should match the intended use of a risk prediction model, we argue that routine use of these indices is not justified. Instead, we recommend decision-theoretic measures to evaluate risk prediction models for applications in which clinically relevant risk thresholds have been established for classifying individuals. In the absence of established risk thresholds, additional research is needed to develop suitable metrics. Copyright © 2017 John Wiley & Sons, Ltd.

Weighing the Evidence of Efficacy of Oral PrEP for HIV Prevention in Women in Southern Africa

As oral tenofovir-based regimens for preexposure prophylaxis (PrEP) are adopted as standard of care for HIV prevention, their utilization in clinical trials among women in southern Africa will require an accurate estimate of oral PrEP efficacy in this population. This information is critical for women in choosing this prevention strategy, and in public health policy making. Estimates of the efficacy of oral PrEP regimens containing tenofovir have varied widely across trials that enrolled women, with some studies reporting high efficacy and others reporting no efficacy. Although poor adherence is strongly associated with lack of efficacy, other factors, such as mode of transmission (sexual vs. parenteral), predominant HIV subtype (C vs. non-C), intensity of exposure, and percentage of stable serodiscordant couples, may also contribute to the variation in efficacy estimates. In this article, we evaluate the evidence for PrEP efficacy in women and propose potential explanations for the observed differences in efficacy among studies. Our review emphasizes the need to continue to refine estimates of efficacy and effectiveness of tenofovir-based oral PrEP so as to best develop the next generation of HIV prevention tools, and to inform public policies directed toward HIV prevention.

Case-only approach to identifying markers predicting treatment effects on the relative risk scale

Retrospectively measuring markers on stored baseline samples from participants in a randomized controlled trial (RCT) may provide high quality evidence as to the value of the markers for treatment selection. Originally developed for approximating gene-environment interactions in the odds ratio scale, the case-only method has recently been advocated for assessing gene-treatment interactions on rare disease endpoints in randomized clinical trials. In this article, the case-only approach is shown to provide a consistent and efficient estimator of marker by treatment interactions and marker-specific treatment effects on the relative risk scale. The prohibitive rare-disease assumption is no longer needed, broadening the utility of the case-only approach. The case-only method is resource-efficient as markers only need to be measured in cases only. It eliminates the need to model the marker's main effect, and can be used with any parametric or nonparametric learning method. The utility of this approach is illustrated by an application to genetic data in the Women's Health Initiative (WHI) hormone therapy trial.

Evaluation of biomarkers for treatment selection using individual participant data from multiple clinical trials

Biomarkers that predict treatment effects may be used to guide treatment decisions, thus improving patient outcomes. A meta-analysis of individual participant data (IPD) is potentially more powerful than a single-study data analysis in evaluating markers for treatment selection. Our study was motivated by the IPD that were collected from 2 randomized controlled trials of hypertension and preeclampsia among pregnant women to evaluate the effect of labor induction over expectant management of the pregnancy in preventing progression to severe maternal disease. The existing literature on statistical methods for biomarker evaluation in IPD meta-analysis have evaluated a marker's performance in terms of its ability to predict risk of disease outcome, which do not directly apply to the treatment selection problem. In this study, we propose a statistical framework for evaluating a marker for treatment selection given IPD from a small number of individual clinical trials. We derive marker-based treatment rules by minimizing the average expected outcome across studies. The application of the proposed methods to the IPD from 2 studies in women with hypertension in pregnancy is presented.

Statistical methods for down-selection of treatment regimens based on multiple endpoints, with application to HIV vaccine trials

Biomarker endpoints measuring vaccine-induced immune responses are essential to HIV vaccine development because of their potential to predict the effect of a vaccine in preventing HIV infection. A vaccine's immune response profile observed in phase I immunogenicity studies is a key factor in determining whether it is advanced for further study in phase II and III efficacy trials. The multiplicity of immune variables and scientific uncertainty in their relative importance, however, pose great challenges to the development of formal algorithms for selecting vaccines to study further. Motivated by the practical need to identify a set of promising vaccines from a pool of candidate regimens for inclusion in an upcoming HIV vaccine efficacy trial, we propose a new statistical framework for the selection of vaccine regimens based on their immune response profile. In particular, we propose superiority and non-redundancy criteria to be achieved in down-selection, and develop novel statistical algorithms that integrate hypothesis testing and ranking for selecting vaccine regimens satisfying these criteria. Performance of the proposed selection algorithms are evaluated through extensive numerical studies. We demonstrate the application of the proposed methods through the comparison of immune responses between several HIV vaccine regimens. The methods are applicable to general down-selection applications in clinical trials.

Predictors of durable immune responses six months after the last vaccination in preventive HIV vaccine trials

BACKGROUND

The evaluation of durable immune responses is important in HIV vaccine research and development. The efficiency of such evaluation could be increased by incorporating predictors of the responses in the statistical analysis. In this paper, we investigated whether and how baseline demographic variables and immune responses measured two weeks after vaccination predicted durable immune responses measured six months later.

METHODS

We included data from seven preventive HIV vaccine regimens evaluated in three clinical trials: a Phase 1 study of four DNA, NYVAC and/or AIDSVAX vaccine regimens (HVTN096), a Phase 2 study of two DNA and/or MVA vaccine regimens (HVTN205), and a Phase 3 study of a single ALVAC/AIDSVAX regimen (RV144). Regularized random forests and linear regression models were used to identify and evaluate predictors of the positivity and magnitude of durable immune responses.

RESULTS

We analyzed 201 vaccine recipients with data from 10 to 127 immune response biomarkers, and 3-5 demographic variables. The best prediction of participants' durable response positivity based on two-week responses rendered up to close-to-perfect accuracy; the best prediction of participants' durable response magnitude rendered correlation coefficients between the observed and predicted responses ranging up to 0.91. Though prediction performances differed among biomarkers, durable immune responses were best predicted by the two-week response level of the same biomarker. Adding demographic information and two-week response levels of different biomarkers provided little or no improvement in the predictions.

CONCLUSIONS

For some biomarkers and for the vaccines we studied, two-week post-vaccination responses can well predict durable responses six months later. Therefore, if immune response durability is only assessed in a sub-sample of vaccine recipients, statistical analyses of durable responses will have increased efficiency by incorporating two-week response data. Further research is needed to generalize the findings to other vaccine regimens and biomarkers. Clinicaltrials.gov identifiers: NCT01799954, NCT00820846, NCT00223080.

Assessing the Clinical Impact of Risk Prediction Models With Decision Curves: Guidance for Correct Interpretation and Appropriate Use

The decision curve is a graphical summary recently proposed for assessing the potential clinical impact of risk prediction biomarkers or risk models for recommending treatment or intervention. It was applied recently in an article in Journal of Clinical Oncology to measure the impact of using a genomic risk model for deciding on adjuvant radiation therapy for prostate cancer treated with radical prostatectomy. We illustrate the use of decision curves for evaluating clinical- and biomarker-based models for predicting a man's risk of prostate cancer, which could be used to guide the decision to biopsy. Decision curves are grounded in a decision-theoretical framework that accounts for both the benefits of intervention and the costs of intervention to a patient who cannot benefit. Decision curves are thus an improvement over purely mathematical measures of performance such as the area under the receiver operating characteristic curve. However, there are challenges in using and interpreting decision curves appropriately. We caution that decision curves cannot be used to identify the optimal risk threshold for recommending intervention. We discuss the use of decision curves for miscalibrated risk models. Finally, we emphasize that a decision curve shows the performance of a risk model in a population in which every patient has the same expected benefit and cost of intervention. If every patient has a personal benefit and cost, then the curves are not useful. If subpopulations have different benefits and costs, subpopulation-specific decision curves should be used. As a companion to this article, we released an R software package called DecisionCurve for making decision curves and related graphics.

Early-Phase Studies of Biomarkers: What Target Sensitivity and Specificity Values Might Confer Clinical Utility?

BACKGROUND

Many cancer biomarker research studies seek to develop markers that can accurately detect or predict future onset of disease. To design and evaluate these studies, one must specify the levels of accuracy sought. However, justified target levels are rarely available.

METHODS

We describe a way to calculate target levels of sensitivity and specificity for a biomarker intended to be applied in a defined clinical context. The calculation requires knowledge of the prevalence or incidence of cases in the clinical population and the ratio of benefit associated with the clinical consequences of a positive biomarker test in cases (true positive) to cost associated with a positive biomarker test in controls (false positive). Guidance is offered on soliciting the cost/benefit ratio. The calculations are based on the longstanding decision theory concept of providing a net benefit on average in the population, and they rely on some assumptions about uniformity of costs and benefits to those tested.

RESULTS

Calculations are illustrated with 3 applications: predicting colon cancer recurrence in stage 1 patients; predicting interval breast cancer (between mammography screenings); and screening for ovarian cancer.

CONCLUSIONS

It is feasible to specify target levels of biomarker performance that enable evaluation of the potential clinical impact of biomarkers in early-phase studies. Nevertheless, biomarkers meeting the criteria should still be tested rigorously in studies that measure the actual impact on patient outcomes of using the biomarker to make clinical decisions.

Selection of HIV vaccine candidates for concurrent testing in an efficacy trial

Phase IIb or III HIV-1 vaccine efficacy trials are generally large and operationally challenging. To mitigate this challenge, the HIV Vaccine Trials Network is designing a Phase IIb efficacy trial accommodating the evaluation of multiple vaccine regimens concurrently. As this efficacy trial would evaluate a limited number of vaccine regimens, there is a need to develop a framework for optimizing the strategic selection of regimens from the large number of vaccine candidates tested in Phase I/IIa trials. In this paper we describe the approaches for the selection process, including the choice of immune response endpoints and the statistical criteria and algorithms. We illustrate the selection approaches using data from HIV-1 vaccine trials.

Abstract IA20: Biomarkers for risk prediction, screening, and early diagnostics

Objective: We developed and validated a hybrid risk classifier combining serum markers and epidemiologic risk factors to identify post-menopausal women at elevated risk for invasive fallopian tube, primary peritoneal, and ovarian epithelial carcinoma. The marker component of the classifier uses HE4 in addition to CA125. We have previously reported that HE4 may be useful as a confirmatory test for early diagnosis of ovarian cancer, when rising CA125 is used as a first-line screen.

Methods: To select epidemiologic risk factors for use in the classifier, Cox proportional hazards analyses were conducted using 74,786 Women's Health Initiative (WHI) Observational Study (OS) participants. To construct a combination classifier, 210 WHI OS cases and 536 matched controls with serum marker measurements were analyzed; validation employed 143 cases and 725 matched controls from the WHI Clinical Trial (CT) with similar data.

Results: Analyses identified a combination risk classifier composed of two elevated-risk groups: 1) women with either CA125 or HE4 exceeding a 98% specificity threshold; and 2) women with intact fallopian tubes, prior use of menopausal hormone therapy for at least two years, and either a first degree relative with breast or ovarian cancer or a personal history of breast cancer. In the WHI OS population, it classified 13% of women as elevated risk, identifying 30% of ovarian cancers diagnosed up to 7.8 years post-enrollment (Hazard Ratio [HR]=2.6, p&lt;0.001). In the WHI CT validation population, it classified 8% of women as elevated risk, identifying 31% of cancers diagnosed within 7 years of enrollment (HR=4.6, p&lt;0.001).

Conclusion: CA125 and HE4 contributed significantly to risk prediction in a classifier combining serum markers with epidemiologic risk factors. The hybrid risk classifier may be useful to identify post-menopausal women who would benefit from timely surgical intervention to prevent epithelial ovarian cancer.

Citation Format: Nicole Urban, Sarah Hawley, Holly Janes, Beth Y. Karlan, Charles W. Drescher, Mary B. Daly, Garnet L. Anderson. Biomarkers for risk prediction, screening, and early diagnostics. [abstract]. In: Proceedings of the AACR Special Conference on Advances in Ovarian Cancer Research: Exploiting Vulnerabilities; Oct 17-20, 2015; Orlando, FL. Philadelphia (PA): AACR; Clin Cancer Res 2016;22(2 Suppl):Abstract nr IA20.