Analysis of cluster-randomized test-negative designs: cluster-level methods

Reanalysis of cluster randomised trial data to account for exposure misclassification using a per-protocol and complier-restricted approach

The intention-to-treat (ITT) analysis of the Applying Wolbachia to Eliminate Dengue (AWED) trial estimated a protective efficacy of 77.1% for participants resident in areas randomised to receive releases of wMel-infected Aedes aegypti mosquitoes, an emerging dengue preventive intervention. The limiting assumptions of ITT analyses in cluster randomised trials and the mobility of mosquitoes and humans across cluster boundaries indicate the primary analysis is likely to underestimate the full public health benefit. Using spatiotemporally-resolved data on the distribution of Wolbachia mosquitoes and on the mobility of AWED participants (n = 6306), we perform complier-restricted and per-protocol re-examinations of the efficacy of the Wolbachia intervention. Increased intervention efficacy was estimated in all analyses by the refined exposure measures. The complier-restricted analysis returned an estimated efficacy of 80.7% (95% CI 65.9, 89.0) and the per-protocol analysis estimated 82.7% (71.7, 88.4) efficacy when comparing participants with an estimated wMel exposure of ≥ 80% compared to those with <20%. These reanalyses demonstrate how human and mosquito movement can lead to underestimation of intervention effects in trials of vector interventions and indicate that the protective efficacy of Wolbachia is even higher than reported in the primary trial results.

Double Negative Control Inference in Test-Negative Design Studies of Vaccine Effectiveness

The test-negative design (TND) has become a standard approach to evaluate vaccine effectiveness against the risk of acquiring infectious diseases in real-world settings, such as Influenza, Rotavirus, Dengue fever, and more recently COVID-19. In a TND study, individuals who experience symptoms and seek care are recruited and tested for the infectious disease which defines cases and controls. Despite TND's potential to reduce unobserved differences in healthcare seeking behavior (HSB) between vaccinated and unvaccinated subjects, it remains subject to various potential biases. First, residual confounding bias may remain due to unobserved HSB, occupation as healthcare worker, or previous infection history. Second, because selection into the TND sample is a common consequence of infection and HSB, collider stratification bias may exist when conditioning the analysis on testing, which further induces confounding by latent HSB. In this paper, we present a novel approach to identify and estimate vaccine effectiveness in the target population by carefully leveraging a pair of negative control exposure and outcome variables to account for potential hidden bias in TND studies. We illustrate our proposed method with extensive simulation and an application to study COVID-19 vaccine effectiveness using data from the University of Michigan Health System.

Assessing the efficacy of male Wolbachia-infected mosquito deployments to reduce dengue incidence in Singapore: study protocol for a cluster-randomized controlled trial

BACKGROUND

Dengue is a severe environmental public health challenge in tropical and subtropical regions. In Singapore, decreasing seroprevalence and herd immunity due to successful vector control has paradoxically led to increased transmission potential of the dengue virus. We have previously demonstrated that incompatible insect technique coupled with sterile insect technique (IIT-SIT), which involves the release of X-ray-irradiated male Wolbachia-infected mosquitoes, reduced the Aedes aegypti population by 98% and dengue incidence by 88%. This novel vector control tool is expected to be able to complement current vector control to mitigate the increasing threat of dengue on a larger scale. We propose a multi-site protocol to study the efficacy of IIT-SIT at reducing dengue incidence.

METHODS/DESIGN

The study is designed as a parallel, two-arm, non-blinded cluster-randomized (CR) controlled trial to be conducted in high-rise public housing estates in Singapore, an equatorial city-state. The aim is to determine whether large-scale deployment of male Wolbachia-infected Ae. aegypti mosquitoes can significantly reduce dengue incidence in intervention clusters. We will use the CR design, with the study area comprising 15 clusters with a total area of 10.9 km2, covering approximately 722,204 residents in 1713 apartment blocks. Eight clusters will be randomly selected to receive the intervention, while the other seven will serve as non-intervention clusters. Intervention efficacy will be estimated through two primary endpoints: (1) odds ratio of Wolbachia exposure distribution (i.e., probability of living in an intervention cluster) among laboratory-confirmed reported dengue cases compared to test-negative controls and (2) laboratory-confirmed reported dengue counts normalized by population size in intervention versus non-intervention clusters.

DISCUSSION

This study will provide evidence from a multi-site, randomized controlled trial for the efficacy of IIT-SIT in reducing dengue incidence. The trial will provide valuable information to estimate intervention efficacy for this novel vector control approach and guide plans for integration into national vector control programs in dengue-endemic settings.

TRIAL REGISTRATION

ClinicalTrials.gov, identifier: NCT05505682 . Registered on 16 August 2022. Retrospectively registered.

Toward evaluation of disseminated effects of medications for opioid use disorder within provider-based clusters using routinely-collected health data

Routinely-collected health data can be employed to emulate a target trial when randomized trial data are not available. Patients within provider-based clusters likely exert and share influence on each other's treatment preferences and subsequent health outcomes and this is known as dissemination or spillover. Extending a framework to replicate an idealized two-stage randomized trial using routinely-collected health data, an evaluation of disseminated effects within provider-based clusters is possible. In this article, we propose a novel application of causal inference methods for dissemination to retrospective cohort studies in administrative claims data and evaluate the impact of the normality of the random effects distribution for the cluster-level propensity score on estimation of the causal parameters. An extensive simulation study was conducted to study the robustness of the methods under different distributions of the random effects. We applied these methods to evaluate baseline prescription for medications for opioid use disorder among a cohort of patients diagnosed with opioid use disorder and adjust for baseline confounders using information obtained from an administrative claims database. We discuss future research directions in this setting to better address unmeasured confounding in the presence of disseminated effects.

Efficacy of Wolbachia-Infected Mosquito Deployments for the Control of Dengue

BACKGROUND

Aedes aegypti mosquitoes infected with the wMel strain of Wolbachia pipientis are less susceptible than wild-type A. aegypti to dengue virus infection.

METHODS

We conducted a cluster-randomized trial involving releases of wMel-infected A. aegypti mosquitoes for the control of dengue in Yogyakarta, Indonesia. We randomly assigned 12 geographic clusters to receive deployments of wMel-infected A. aegypti (intervention clusters) and 12 clusters to receive no deployments (control clusters). All clusters practiced local mosquito-control measures as usual. A test-negative design was used to assess the efficacy of the intervention. Patients with acute undifferentiated fever who presented to local primary care clinics and were 3 to 45 years of age were recruited. Laboratory testing was used to identify participants who had virologically confirmed dengue (VCD) and those who were test-negative controls. The primary end point was symptomatic VCD of any severity caused by any dengue virus serotype.

RESULTS

After successful introgression of wMel into the intervention clusters, 8144 participants were enrolled; 3721 lived in intervention clusters, and 4423 lived in control clusters. In the intention-to-treat analysis, VCD occurred in 67 of 2905 participants (2.3%) in the intervention clusters and in 318 of 3401 (9.4%) in the control clusters (aggregate odds ratio for VCD, 0.23; 95% confidence interval [CI], 0.15 to 0.35; P = 0.004). The protective efficacy of the intervention was 77.1% (95% CI, 65.3 to 84.9) and was similar against the four dengue virus serotypes. The incidence of hospitalization for VCD was lower among participants who lived in intervention clusters (13 of 2905 participants [0.4%]) than among those who lived in control clusters (102 of 3401 [3.0%]) (protective efficacy, 86.2%; 95% CI, 66.2 to 94.3).

CONCLUSIONS

Introgression of wMel into A. aegypti populations was effective in reducing the incidence of symptomatic dengue and resulted in fewer hospitalizations for dengue among the participants. (Funded by the Tahija Foundation and others; AWED ClinicalTrials.gov number, NCT03055585; Indonesia Registry number, INA-A7OB6TW.).

The TIRS trial: protocol for a cluster randomized controlled trial assessing the efficacy of preventive targeted indoor residual spraying to reduce Aedes-borne viral illnesses in Merida, Mexico

BACKGROUND

Current urban vector control strategies have failed to contain dengue epidemics and to prevent the global expansion of Aedes-borne viruses (ABVs: dengue, chikungunya, Zika). Part of the challenge in sustaining effective ABV control emerges from the paucity of evidence regarding the epidemiological impact of any Aedes control method. A strategy for which there is limited epidemiological evidence is targeted indoor residual spraying (TIRS). TIRS is a modification of classic malaria indoor residual spraying that accounts for Aedes aegypti resting behavior by applying residual insecticides on exposed lower sections of walls (< 1.5 m), under furniture, and on dark surfaces.

METHODS/DESIGN

We are pursuing a two-arm, parallel, unblinded, cluster randomized controlled trial to quantify the overall efficacy of TIRS in reducing the burden of laboratory-confirmed ABV clinical disease (primary endpoint). The trial will be conducted in the city of Merida, Yucatan State, Mexico (population ~ 1million), where we will prospectively follow 4600 children aged 2-15 years at enrollment, distributed in 50 clusters of 5 × 5 city blocks each. Clusters will be randomly allocated (n = 25 per arm) using covariate-constrained randomization. A "fried egg" design will be followed, in which all blocks of the 5 × 5 cluster receive the intervention, but all sampling to evaluate the epidemiological and entomological endpoints will occur in the "yolk," the center 3 × 3 city blocks of each cluster. TIRS will be implemented as a preventive application (~ 1-2 months prior to the beginning of the ABV season). Active monitoring for symptomatic ABV illness will occur through weekly household visits and enhanced surveillance. Annual sero-surveys will be performed after each transmission season and entomological evaluations of Ae. aegypti indoor abundance and ABV infection rates monthly during the period of active surveillance. Epidemiological and entomological evaluation will continue for up to three transmission seasons.

DISCUSSION

The findings from this study will provide robust epidemiological evidence of the efficacy of TIRS in reducing ABV illness and infection. If efficacious, TIRS could drive a paradigm shift in Aedes control by considering Ae. aegypti behavior to guide residual insecticide applications and changing deployment to preemptive control (rather than in response to symptomatic cases), two major enhancements to existing practice.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04343521 . Registered on 13 April 2020. The protocol also complies with the WHO International Clinical Trials Registry Platform (ICTRP) (Additional file 1).

PRIMARY SPONSOR

National Institutes of Health, National Institute of Allergy and Infectious Diseases (NIH/NIAID).

Relative effectiveness of high dose versus standard dose influenza vaccines in older adult outpatients over four seasons, 2015-16 to 2018-19

BACKGROUND

New influenza vaccine formulations are designed to improve vaccine effectiveness and protect those most vulnerable to infection. High dose trivalent inactivated influenza vaccine (HD-IIV3), licensed for ages ≥65 years, produces greater antibody responses and efficacy in clinical trials, but post-licensure vaccine effectiveness (VE) compared to standard dose (SD-IIV3/4) vaccine remains an open question.

METHODS

Using a test-negative, case control design and propensity analyses to adjust for confounding, US Influenza VE Network data from the 2015-2016 through 2018-2019 seasons were analyzed to determine relative VE (rVE) between HD-IIV3 and SD-IIV3/4 among outpatients ≥65 years old presenting with acute respiratory illness. Influenza vaccination status was derived from electronic medical records and immunization registries.

RESULTS

Among 3861 enrollees, 2993 (78%) were vaccinated; 1573 (53%) received HD-IIV3 and 1420 (47%) received SD-IIV3/4. HD-IIV3 recipients differed from SD-IIV3/4 recipients by race, previous vaccination, number of outpatient visits in the previous year and timing of vaccination, and were balanced in the propensity model except the timing of vaccination. Compared with no vaccination, significant protection against any influenza A was observed from both HD-IIV3 (VE = 29%; 95%CI = 10%, 44%) and SD-IIV3/4 (VE = 24%; 95%CI = 5%, 39%); rVE = 18% (95%CI = 0%, 33%, SD as referent). When stratified by virus type, against A/H1N1, HD-IIV3 VE was 30% (95%CI = -7%, 54%), SD-IIV3/4 VE was 40% (95%CI = 10%, 61%), and rVE = -32%; (95%CI = -94%, 11%); Against A/H3N2, HD-IIV3 VE was 31% (95%CI = 9%, 47%), SD-IIV3/4 VE was 19% (95%CI = -5%, 37%), and rVE = 27%; (95% CI = 9%, 42%).

CONCLUSIONS

Among adults ≥65 years of age, recipients of standard and high dose influenza vaccines differed significantly in their characteristics. After adjusting for these differences, high dose vaccine offered more protection against A/H3N2 and borderline significant protection against all influenza A requiring outpatient care during the 2015-2018 influenza seasons.

Analysis of counts for cluster randomized trials: Negative controls and test-negative designs

In cluster randomized trials (CRTs), the outcome of interest is often a count at the cluster level. This occurs, for example, in evaluating an intervention with the outcome being the number of infections of a disease such as HIV or dengue or the number of hospitalizations in the cluster. Standard practice analyzes these counts through cluster outcome rates using an appropriate denominator (eg, population size). However, such denominators are sometimes unknown, particularly when the counts depend on a passive community surveillance system. We consider direct comparison of the counts without knowledge of denominators, relying on randomization to balance denominators. We also focus on permutation tests to allow for small numbers of randomized clusters. However, such approaches are subject to bias when there is differential ascertainment of counts across arms, a situation that may occur in CRTs that cannot implement blinded interventions. We suggest the use of negative control counts as a method to remove, or reduce, this bias, discussing the key properties necessary for an effective negative control. A current example of such a design is the recent extension of test-negative designs to CRTs testing community-level interventions. Via simulation, we compare the performance of new and standard estimators based on CRTs with negative controls to approaches that only use the original counts. When there is no differential ascertainment by intervention arm, the count-only approaches perform comparably to those using debiasing negative controls. However, under even modest differential ascertainment, the count-only estimators are no longer reliable.

The Use of Test-negative Controls to Monitor Vaccine Effectiveness: A Systematic Review of Methodology

BACKGROUND

The test-negative design is an increasingly popular approach for estimating vaccine effectiveness (VE) due to its efficiency. This review aims to examine published test-negative design studies of VE and to explore similarities and differences in methodological choices for different diseases and vaccines.

METHODS

We conducted a systematic search on PubMed, Web of Science, and Medline, for studies reporting the effectiveness of any vaccines using a test-negative design. We screened titles and abstracts and reviewed full texts to identify relevant articles. We created a standardized form for each included article to extract information on the pathogen of interest, vaccine(s) being evaluated, study setting, clinical case definition, choices of cases and controls, and statistical approaches used to estimate VE.

RESULTS

We identified a total of 348 articles, including studies on VE against influenza virus (n = 253), rotavirus (n = 48), pneumococcus (n = 24), and nine other pathogens. Clinical case definitions used to enroll patients were similar by pathogens of interest but the sets of symptoms that defined them varied substantially. Controls could be those testing negative for the pathogen of interest, those testing positive for nonvaccine type of the pathogen of interest, or a subset of those testing positive for alternative pathogens. Most studies controlled for age, calendar time, and comorbidities.

CONCLUSIONS

Our review highlights similarities and differences in the application of the test-negative design that deserve further examination. If vaccination reduces disease severity in breakthrough infections, particular care must be taken in interpreting vaccine effectiveness estimates from test-negative design studies.

Long-term circulation of Zika virus in Thailand: an observational study

BACKGROUND

Little is known about the historical and current risk of Zika virus infection in southeast Asia, where the mosquito vector is widespread and other arboviruses circulate endemically. Centralised Zika virus surveillance began in Thailand in January, 2016. We assessed the long-term circulation of Zika virus in Thailand.

METHODS

In this observational study, we analysed data from individuals with suspected Zika virus infection who presented at hospitals throughout the country and had biological samples (serum, plasma, or urine) tested for confirmation with PCR at the National Institute of Health laboratories in Bangkok. We analysed the spatial and age distribution of cases, and constructed time-resolved phylogenetic trees using genomes from Thailand and elsewhere to estimate when Zika virus was first introduced.

FINDINGS

Of the 3089 samples from 1717 symptomatic individuals tested between January, 2016, and December, 2017, 368 were confirmed to have Zika virus infection. Cases of Zika virus infection were reported throughout the year, and from 29 of the 76 Thai provinces. Individuals had 2·8 times (95% CI 2·3-3·6) the odds of testing positive for Zika virus infection if they came from the same district and were sick within the same year of a person with a confirmed infection relative to the odds of testing positive anywhere, consistent with focal transmission. The probability of cases being younger than 10 years was 0·99 times (0·72-1·30) the probability of being that age in the underlying population. This probability rose to 1·62 (1·33-1·92) among those aged 21-30 years and fell to 0·53 (0·40-0·66) for those older than 50 years. This age distribution is consistent with that observed in the Zika virus epidemic in Colombia. Phylogenetic reconstructions suggest persistent circulation within Thailand since at least 2002.

INTERPRETATION

Our evidence shows that Zika virus has circulated at a low but sustained level for at least 16 years, suggesting that Zika virus can adapt to persistent endemic transmission. Health systems need to adapt to cope with regular occurrences of the severe complications associated with infection.

FUNDING

European Research Council, National Science Foundation, and National Institutes of Health.

Measurement of Vaccine Direct Effects Under the Test-Negative Design

Test-negative designs are commonplace in assessments of influenza vaccination effectiveness, estimating this value from the exposure odds ratio of vaccination among individuals treated for acute respiratory illness who test positive for influenza virus infection. This approach is widely believed to recover the vaccine direct effect by correcting for differential health-care-seeking behavior among vaccinated and unvaccinated persons. However, the relationship of the measured odds ratio to true vaccine effectiveness is poorly understood. We derived the odds ratio under circumstances of real-world test-negative studies. The odds ratio recovers the vaccine direct effect when 2 conditions are met: 1) Individuals' vaccination decisions are uncorrelated with exposure or susceptibility to the test-positive or test-negative conditions, and 2) vaccination confers "all-or-nothing" protection (whereby certain individuals have no protection while others are perfectly protected). Biased effect-size estimates arise if either condition is unmet. Such bias might suggest misleading associations of vaccine effectiveness with time since vaccination or the force of infection of influenza. The test-negative design could also fail to correct for differential health-care-seeking behavior among vaccinated and unvaccinated persons without stringent criteria for enrollment and testing. Our findings demonstrate a need to reassess how data from test-negative studies can inform policy decisions.

Cluster-Randomized Test-Negative Design Trials: A Novel and Efficient Method to Assess the Efficacy of Community-Level Dengue Interventions

Cluster-randomized controlled trials are the gold standard for assessing efficacy of community-level interventions, such as vector-control strategies against dengue. We describe a novel cluster-randomized trial methodology with a test-negative design (CR-TND), which offers advantages over traditional approaches. This method uses outcome-based sampling of patients presenting with a syndrome consistent with the disease of interest, who are subsequently classified as test-positive cases or test-negative controls on the basis of diagnostic testing. We used simulations of a cluster trial to demonstrate validity of efficacy estimates under the test-negative approach. We demonstrated that, provided study arms are balanced for both test-negative and test-positive illness at baseline and that other test-negative design assumptions are met, the efficacy estimates closely match true efficacy. Analytical considerations for an odds ratio-based effect estimate arising from clustered data and potential approaches to analysis are also discussed briefly. We concluded that application of the test-negative design to certain cluster-randomized trials could increase their efficiency and ease of implementation.

The AWED trial (Applying Wolbachia to Eliminate Dengue) to assess the efficacy of Wolbachia-infected mosquito deployments to reduce dengue incidence in Yogyakarta, Indonesia: study protocol for a cluster randomised controlled trial

Background

Dengue and other arboviruses transmitted by Aedes aegypti mosquitoes, including Zika and chikungunya, present an increasing public health challenge in tropical regions. Current vector control strategies have failed to curb disease transmission, but continue to be employed despite the absence of robust evidence for their effectiveness or optimal implementation. The World Mosquito Program has developed a novel approach to arbovirus control using Ae. aegypti stably transfected with Wolbachia bacterium, with a significantly reduced ability to transmit dengue, Zika and chikungunya in laboratory experiments. Modelling predicts this will translate to local elimination of dengue in most epidemiological settings. This study protocol describes the first trial to measure the efficacy of Wolbachia in reducing dengue virus transmission in the field.

Methods/design

The study is a parallel, two-arm, non-blinded cluster randomised controlled trial conducted in a single site in Yogyakarta, Indonesia. The aim is to determine whether large-scale deployment of Wolbachia-infected Ae. aegypti mosquitoes leads to a measurable reduction in dengue incidence in treated versus untreated areas. The primary endpoint is symptomatic, virologically confirmed dengue virus infection of any severity. The 26 km² study area was subdivided into 24 contiguous clusters, allocated randomly 1:1 to receive Wolbachia deployments or no intervention. We use a novel epidemiological study design, the cluster-randomised test-negative design trial, in which dengue cases and arbovirus-negative controls are sampled concurrently from among febrile patients presenting to a network of primary care clinics, with case or control status classified retrospectively based on the results of laboratory diagnostic testing. Efficacy is estimated from the odds ratio of Wolbachia exposure distribution (probability of living in a Wolbachia-treated area) among virologically confirmed dengue cases compared to test-negative controls. A secondary per-protocol analysis allows for individual Wolbachia exposure levels to be assessed to account for movements outside the cluster and the heterogeneity in local Wolbachia prevalence among treated clusters.

Discussion

The findings from this study will provide the first experimental evidence for the efficacy of Wolbachia in reducing dengue incidence. Together with observational evidence that is accumulating from pragmatic deployments of Wolbachia in other field sites, this will provide valuable data to estimate the effectiveness of this novel approach to arbovirus control, inform future cost-effectiveness estimates, and guide plans for large-scale deployments in other endemic settings.

Trial registration

ClinicalTrials.gov, identifier: NCT03055585. Registered on 14 February 2017.

Electronic supplementary material

The online version of this article (10.1186/s13063-018-2670-z) contains supplementary material, which is available to authorized users.