Cluster-Randomized Test-Negative Design Trials: A Novel and Efficient Method to Assess the Efficacy of Community-Level Dengue Interventions

Assessing Wolbachia-mediated sterility for dengue control: emulation of a cluster-randomized target trial in Singapore

BACKGROUND

Matings between male Aedes aegypti mosquitoes infected with wAlbB strain of Wolbachia and wildtype females yield non-viable eggs. We evaluated the efficacy of releasing wAlbB-infected Ae. aegypti male mosquitoes to suppress dengue.

METHODS

We specified the protocol of a two-arm cluster-randomized test-negative controlled trial (cRCT) and emulated it using a nationally representative test-negative/positive database of individuals reporting for febrile illness to any public hospital, general practitioner or polyclinic. We retrospectively built a cohort of individuals who reside in Wolbachia locations vs a comparator control group who do not reside in Wolbachia locations, using a nationally representative database of all individuals whom report for febrile illness and were tested for dengue at the Environmental Health Institute/hospital laboratories/commercial diagnostic laboratories, through general practitioner clinic, polyclinic or public/private hospital from epidemiological week (EW) 1 2019 to EW26 2022. We emulated a constrained randomization protocol used in cRCTs to balance dengue risk between intervention and control arms in the pre-intervention period. We used the inverse probability weighting approach to further balance the intervention and control groups using a battery of algorithmically selected sociodemographic, environmental and anthropogenic variables. Intention-to-treat analyses were conducted to estimate the risk reduction of dengue given Wolbachia exposure.

RESULTS

Intention-to-treat analyses revealed that, compared with controls, Wolbachia releases for 3, 6 and ≥12 months was associated to 47% (95% confidence interval: 25-69%), 44% (33-77%) and 61% (38-78%) protective efficacy against dengue, respectively. When exposed to ≥12 months of Wolbachia releases, protective efficacies ranged from 49% (13-72%) to 77% (60-94%) across years. The proportion of virologically confirmed dengue cases was lower overall in the intervention arm. Protective efficacies were found across all years, age and sex subgroups, with higher durations of Wolbachia exposure associated to greater risk reductions of dengue.

CONCLUSION

Results demonstrated that Wolbachia-mediated sterility can strengthen dengue control in tropical cities, where dengue burden is the greatest.

Hypothesis testing and sample size considerations for the test-negative design

The test-negative design (TND) is an observational study design to evaluate vaccine effectiveness (VE) that enrolls individuals receiving diagnostic testing for a target disease as part of routine care. VE is estimated as one minus the adjusted odds ratio of testing positive versus negative comparing vaccinated and unvaccinated patients. Although the TND is related to case-control studies, it is distinct in that the ratio of test-positive cases to test-negative controls is not typically pre-specified. For both types of studies, sparse cells are common when vaccines are highly effective. We consider the implications of these features on power for the TND. We use simulation studies to explore three hypothesis-testing procedures and associated sample size calculations for case-control and TND studies. These tests, all based on a simple logistic regression model, are a standard Wald test, a continuity-corrected Wald test, and a score test. The Wald test performs poorly in both case-control and TND when VE is high because the number of vaccinated test-positive cases can be low or zero. Continuity corrections help to stabilize the variance but induce bias. We observe superior performance with the score test as the variance is pooled under the null hypothesis of no group differences. We recommend using a score-based approach to design and analyze both case-control and TND. We propose a modification to the TND score sample size to account for additional variability in the ratio of controls over cases. This work enhances our understanding of the data generating mechanism in a test-negative design (TND) and how it is distinct from that of a case-control study due to its passive recruitment of controls.

Considerations for first field trials of low-threshold gene drive for malaria vector control

Sustainable reductions in African malaria transmission require innovative tools for mosquito control. One proposal involves the use of low-threshold gene drive in Anopheles vector species, where a 'causal pathway' would be initiated by (i) the release of a gene drive system in target mosquito vector species, leading to (ii) its transmission to subsequent generations, (iii) its increase in frequency and spread in target mosquito populations, (iv) its simultaneous propagation of a linked genetic trait aimed at reducing vectorial capacity for Plasmodium, and (v) reduced vectorial capacity for parasites in target mosquito populations as the gene drive system reaches fixation in target mosquito populations, causing (vi) decreased malaria incidence and prevalence. Here the scope, objectives, trial design elements, and approaches to monitoring for initial field releases of such gene dive systems are considered, informed by the successful implementation of field trials of biological control agents, as well as other vector control tools, including insecticides, Wolbachia, larvicides, and attractive-toxic sugar bait systems. Specific research questions to be addressed in initial gene drive field trials are identified, and adaptive trial design is explored as a potentially constructive and flexible approach to facilitate testing of the causal pathway. A fundamental question for decision-makers for the first field trials will be whether there should be a selective focus on earlier points of the pathway, such as genetic efficacy via measurement of the increase in frequency and spread of the gene drive system in target populations, or on wider interrogation of the entire pathway including entomological and epidemiological efficacy. How and when epidemiological efficacy will eventually be assessed will be an essential consideration before decisions on any field trial protocols are finalized and implemented, regardless of whether initial field trials focus exclusively on the measurement of genetic efficacy, or on broader aspects of the causal pathway. Statistical and modelling tools are currently under active development and will inform such decisions on initial trial design, locations, and endpoints. Collectively, the considerations here advance the realization of developer ambitions for the first field trials of low-threshold gene drive for malaria vector control within the next 5 years.

Hypothesis testing and sample size considerations for the test-negative design

The test-negative design (TND) is an observational study design to evaluate vaccine effectiveness (VE) that enrolls individuals receiving diagnostic testing for a target disease as part of routine care. VE is estimated as one minus the adjusted odds ratio of testing positive versus negative comparing vaccinated and unvaccinated patients. Although the TND is related to case-control studies, it is distinct in that the ratio of test-positive cases to test-negative controls is not typically pre-specified. For both types of studies, sparse cells are common when vaccines are highly effective. We consider the implications of these features on power for the TND. We use simulation studies to explore three hypothesis-testing procedures and associated sample size calculations for case-control and TND studies. These tests, all based on a simple logistic regression model, are a standard Wald test, a continuity-corrected Wald test, and a score test. The Wald test performs poorly in both case-control and TND when VE is high because the number of vaccinated test-positive cases can be low or zero. Continuity corrections help to stabilize the variance but induce bias. We observe superior performance with the score test as the variance is pooled under the null hypothesis of no group differences. We recommend using a score-based approach to design and analyze both case-control and TND. We propose a modification to the TND score sample size to account for additional variability in the ratio of controls over cases. This work expands our understanding of the data mechanisms of the TND.

Programmatic Effectiveness of a Pediatric Typhoid Conjugate Vaccine Campaign in Navi Mumbai, India

BACKGROUND

The World Health Organization recommends vaccines for prevention and control of typhoid fever, especially where antimicrobial-resistant typhoid circulates. In 2018, the Navi Mumbai Municipal Corporation (NMMC) implemented a typhoid conjugate vaccine (TCV) campaign. The campaign targeted all children aged 9 months through 14 years within NMMC boundaries (approximately 320 000 children) over 2 vaccination phases. The phase 1 campaign occurred from 14 July 2018 through 25 August 2018 (71% coverage, approximately 113 420 children). We evaluated the phase 1 campaign's programmatic effectiveness in reducing typhoid cases at the community level.

METHODS

We established prospective, blood culture-based surveillance at 6 hospitals in Navi Mumbai and offered blood cultures to children who presented with fever ≥3 days. We used a cluster-randomized (by administrative boundary) test-negative design to estimate the effectiveness of the vaccination campaign on pediatric typhoid cases. We matched test-positive, culture-confirmed typhoid cases with up to 3 test-negative, culture-negative controls by age and date of blood culture and assessed community vaccine campaign phase as an exposure using conditional logistic regression.

RESULTS

Between 1 September 2018 and 31 March 2021, we identified 81 typhoid cases and matched these with 238 controls. Cases were 0.44 times as likely to live in vaccine campaign communities (programmatic effectiveness, 56%; 95% confidence interval [CI], 25% to 74%; P = .002). Cases aged ≥5 years were 0.37 times as likely (95% CI, .19 to .70; P = .002) and cases during the first year of surveillance were 0.30 times as likely (95% CI, .14 to .64; P = .002) to live in vaccine campaign communities.

CONCLUSIONS

Our findings support the use of TCV mass vaccination campaigns as effective population-based tools to combat typhoid fever.

Assessing the efficacy of male Wolbachia-infected mosquito deployments to reduce dengue incidence in Singapore: study protocol for a cluster-randomized controlled trial

BACKGROUND

Dengue is a severe environmental public health challenge in tropical and subtropical regions. In Singapore, decreasing seroprevalence and herd immunity due to successful vector control has paradoxically led to increased transmission potential of the dengue virus. We have previously demonstrated that incompatible insect technique coupled with sterile insect technique (IIT-SIT), which involves the release of X-ray-irradiated male Wolbachia-infected mosquitoes, reduced the Aedes aegypti population by 98% and dengue incidence by 88%. This novel vector control tool is expected to be able to complement current vector control to mitigate the increasing threat of dengue on a larger scale. We propose a multi-site protocol to study the efficacy of IIT-SIT at reducing dengue incidence.

METHODS/DESIGN

The study is designed as a parallel, two-arm, non-blinded cluster-randomized (CR) controlled trial to be conducted in high-rise public housing estates in Singapore, an equatorial city-state. The aim is to determine whether large-scale deployment of male Wolbachia-infected Ae. aegypti mosquitoes can significantly reduce dengue incidence in intervention clusters. We will use the CR design, with the study area comprising 15 clusters with a total area of 10.9 km2, covering approximately 722,204 residents in 1713 apartment blocks. Eight clusters will be randomly selected to receive the intervention, while the other seven will serve as non-intervention clusters. Intervention efficacy will be estimated through two primary endpoints: (1) odds ratio of Wolbachia exposure distribution (i.e., probability of living in an intervention cluster) among laboratory-confirmed reported dengue cases compared to test-negative controls and (2) laboratory-confirmed reported dengue counts normalized by population size in intervention versus non-intervention clusters.

DISCUSSION

This study will provide evidence from a multi-site, randomized controlled trial for the efficacy of IIT-SIT in reducing dengue incidence. The trial will provide valuable information to estimate intervention efficacy for this novel vector control approach and guide plans for integration into national vector control programs in dengue-endemic settings.

TRIAL REGISTRATION

ClinicalTrials.gov, identifier: NCT05505682 . Registered on 16 August 2022. Retrospectively registered.

Efficacy of a spatial repellent for control of Aedes-borne virus transmission: A cluster-randomized trial in Iquitos, Peru

Vector interventions are needed for Aedes-borne viral (ABV) disease prevention (dengue, Zika, chikungunya, and yellow fever), but their application is hindered by the lack of evidence proving they prevent infection or disease. We report conclusive statistical evidence from a pre-planned, prospective cluster-randomized, controlled clinical trial (cRCT) of protective efficacy (34.1% hazard estimate) against human ABV infection by a spatial repellent; a chemical-based intervention category currently under World Health Organization review. Results from our ABV study will help guide public health authorities responsible for operational management and worldwide ABV disease control and incentivize new strategies for disease prevention.

Over half the world’s population is at risk for viruses transmitted by Aedes mosquitoes, such as dengue and Zika. The primary vector, Aedes aegypti, thrives in urban environments. Despite decades of effort, cases and geographic range of Aedes-borne viruses (ABVs) continue to expand. Rigorously proven vector control interventions that measure protective efficacy against ABV diseases are limited to Wolbachia in a single trial in Indonesia and do not include any chemical intervention. Spatial repellents, a new option for efficient deployment, are designed to decrease human exposure to ABVs by releasing active ingredients into the air that disrupt mosquito–human contact. A parallel, cluster-randomized controlled trial was conducted in Iquitos, Peru, to quantify the impact of a transfluthrin-based spatial repellent on human ABV infection. From 2,907 households across 26 clusters (13 per arm), 1,578 participants were assessed for seroconversion (primary endpoint) by survival analysis. Incidence of acute disease was calculated among 16,683 participants (secondary endpoint). Adult mosquito collections were conducted to compare Ae. aegypti abundance, blood-fed rate, and parity status through mixed-effect difference-in-difference analyses. The spatial repellent significantly reduced ABV infection by 34.1% (one-sided 95% CI lower limit, 6.9%; one-sided P value = 0.0236, z = 1.98). Aedes aegypti abundance and blood-fed rates were significantly reduced by 28.6 (95% CI 24.1%, ∞); z = −9.11) and 12.4% (95% CI 4.2%, ∞); z = −2.43), respectively. Our trial provides conclusive statistical evidence from an appropriately powered, preplanned cluster-randomized controlled clinical trial of the impact of a chemical intervention, in this case a spatial repellent, to reduce the risk of ABV transmission compared to a placebo.

EVITA Dengue: a cluster-randomized controlled trial to EValuate the efficacy of Wolbachia-InfecTed Aedes aegypti mosquitoes in reducing the incidence of Arboviral infection in Brazil

BACKGROUND

Arboviruses transmitted by Aedes aegypti including dengue, Zika, and chikungunya are a major global health problem, with over 2.5 billion at risk for dengue alone. There are no licensed antivirals for these infections, and safe and effective vaccines are not yet widely available. Thus, prevention of arbovirus transmission by vector modification is a novel approach being pursued by multiple researchers. However, the field needs high-quality evidence derived from randomized, controlled trials upon which to base the implementation and maintenance of vector control programs. Here, we report the EVITA Dengue trial design (DMID 17-0111), which assesses the efficacy in decreasing arbovirus transmission of an innovative approach developed by the World Mosquito Program for vector modification of Aedes mosquitoes by Wolbachia pipientis.

METHODS

DMID 17-0111 is a cluster-randomized trial in Belo Horizonte, Brazil, with clusters defined by primary school catchment areas. Clusters (n = 58) will be randomized 1:1 to intervention (release of Wolbachia-infected Aedes aegypti mosquitoes) vs. control (no release). Standard vector control activities (i.e., insecticides and education campaigns for reduction of mosquito breeding sites) will continue as per current practice in the municipality. Participants (n = 3480, 60 per cluster) are children aged 6-11 years enrolled in the cluster-defining school and living within the cluster boundaries who will undergo annual serologic surveillance for arboviral infection. The primary objective is to compare sero-incidence of arboviral infection between arms.

DISCUSSION

DMID 17-0111 aims to determine the efficacy of Wolbachia-infected mosquito releases in reducing human infections by arboviruses transmitted by Aedes aegypti and will complement the mounting evidence for this method from large-scale field releases and ongoing trials. The trial also represents a critical step towards robustness and rigor for how vector control methods are assessed, including the simultaneous measurement and correlation of entomologic and epidemiologic outcomes. Data from this trial will inform further the development of novel vector control methods.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04514107 . Registered on 17 August 2020 Primary sponsor: National Institute of Health, National Institute of Allergy and Infectious Diseases.

Potential test-negative design study bias in outbreak settings: application to Ebola vaccination in Democratic Republic of Congo

BACKGROUND

Infectious disease outbreaks present unique challenges to study designs for vaccine evaluation. Test-negative design (TND) studies have previously been used to estimate vaccine effectiveness and have been proposed for Ebola virus disease (EVD) vaccines. However, there are key differences in how cases and controls are recruited during outbreaks and pandemics of novel pathogens, whcih have implications for the reliability of effectiveness estimates using this design.

METHODS

We use a modelling approach to quantify TND bias for a prophylactic vaccine under varying study and epidemiological scenarios. Our model accounts for heterogeneity in vaccine distribution and for two potential routes to testing and recruitment into the study: self-reporting and contact-tracing. We derive conventional and hybrid TND estimators for this model and suggest ways to translate public health response data into the parameters of the model.

RESULTS

Using a conventional TND study, our model finds biases in vaccine effectiveness estimates. Bias arises due to differential recruitment from self-reporting and contact-tracing, and due to clustering of vaccination. We estimate the degree of bias when recruitment route is not available, and propose a study design to eliminate the bias if recruitment route is recorded.

CONCLUSIONS

Hybrid TND studies can resolve the design bias with conventional TND studies applied to outbreak and pandemic response testing data, if those efforts collect individuals' routes to testing. Without route to testing, other epidemiological data will be required to estimate the magnitude of potential bias in a conventional TND study. Since these studies may need to be conducted retrospectively, public health responses should obtain these data, and generic protocols for outbreak and pandemic response studies should emphasize the need to record routes to testing.

Efficacy of Wolbachia-Infected Mosquito Deployments for the Control of Dengue

BACKGROUND

Aedes aegypti mosquitoes infected with the wMel strain of Wolbachia pipientis are less susceptible than wild-type A. aegypti to dengue virus infection.

METHODS

We conducted a cluster-randomized trial involving releases of wMel-infected A. aegypti mosquitoes for the control of dengue in Yogyakarta, Indonesia. We randomly assigned 12 geographic clusters to receive deployments of wMel-infected A. aegypti (intervention clusters) and 12 clusters to receive no deployments (control clusters). All clusters practiced local mosquito-control measures as usual. A test-negative design was used to assess the efficacy of the intervention. Patients with acute undifferentiated fever who presented to local primary care clinics and were 3 to 45 years of age were recruited. Laboratory testing was used to identify participants who had virologically confirmed dengue (VCD) and those who were test-negative controls. The primary end point was symptomatic VCD of any severity caused by any dengue virus serotype.

RESULTS

After successful introgression of wMel into the intervention clusters, 8144 participants were enrolled; 3721 lived in intervention clusters, and 4423 lived in control clusters. In the intention-to-treat analysis, VCD occurred in 67 of 2905 participants (2.3%) in the intervention clusters and in 318 of 3401 (9.4%) in the control clusters (aggregate odds ratio for VCD, 0.23; 95% confidence interval [CI], 0.15 to 0.35; P = 0.004). The protective efficacy of the intervention was 77.1% (95% CI, 65.3 to 84.9) and was similar against the four dengue virus serotypes. The incidence of hospitalization for VCD was lower among participants who lived in intervention clusters (13 of 2905 participants [0.4%]) than among those who lived in control clusters (102 of 3401 [3.0%]) (protective efficacy, 86.2%; 95% CI, 66.2 to 94.3).

CONCLUSIONS

Introgression of wMel into A. aegypti populations was effective in reducing the incidence of symptomatic dengue and resulted in fewer hospitalizations for dengue among the participants. (Funded by the Tahija Foundation and others; AWED ClinicalTrials.gov number, NCT03055585; Indonesia Registry number, INA-A7OB6TW.).

Temporal Confounding in the Test-Negative Design

In the test-negative design, routine testing at health-care facilities is leveraged to estimate the effectiveness of an intervention such as a vaccine. The odds of vaccination for individuals who test positive for a target pathogen is compared with the odds of vaccination for individuals who test negative for that pathogen, adjusting for key confounders. The design is rapidly growing in popularity, but many open questions remain about its properties. In this paper, we examine temporal confounding by generalizing derivations to allow for time-varying vaccine status, including out-of-season controls, and open populations. We confirm that calendar time is an important confounder when vaccine status varies during the study. We demonstrate that, where time is not a confounder, including out-of-season controls can improve precision. We generalize these results to open populations. We use our theoretical findings to interpret 3 recent papers utilizing the test-negative design. Through careful examination of the theoretical properties of this study design, we provide key insights that can directly inform the implementation and analysis of future test-negative studies.

The TIRS trial: protocol for a cluster randomized controlled trial assessing the efficacy of preventive targeted indoor residual spraying to reduce Aedes-borne viral illnesses in Merida, Mexico

BACKGROUND

Current urban vector control strategies have failed to contain dengue epidemics and to prevent the global expansion of Aedes-borne viruses (ABVs: dengue, chikungunya, Zika). Part of the challenge in sustaining effective ABV control emerges from the paucity of evidence regarding the epidemiological impact of any Aedes control method. A strategy for which there is limited epidemiological evidence is targeted indoor residual spraying (TIRS). TIRS is a modification of classic malaria indoor residual spraying that accounts for Aedes aegypti resting behavior by applying residual insecticides on exposed lower sections of walls (< 1.5 m), under furniture, and on dark surfaces.

METHODS/DESIGN

We are pursuing a two-arm, parallel, unblinded, cluster randomized controlled trial to quantify the overall efficacy of TIRS in reducing the burden of laboratory-confirmed ABV clinical disease (primary endpoint). The trial will be conducted in the city of Merida, Yucatan State, Mexico (population ~ 1million), where we will prospectively follow 4600 children aged 2-15 years at enrollment, distributed in 50 clusters of 5 × 5 city blocks each. Clusters will be randomly allocated (n = 25 per arm) using covariate-constrained randomization. A "fried egg" design will be followed, in which all blocks of the 5 × 5 cluster receive the intervention, but all sampling to evaluate the epidemiological and entomological endpoints will occur in the "yolk," the center 3 × 3 city blocks of each cluster. TIRS will be implemented as a preventive application (~ 1-2 months prior to the beginning of the ABV season). Active monitoring for symptomatic ABV illness will occur through weekly household visits and enhanced surveillance. Annual sero-surveys will be performed after each transmission season and entomological evaluations of Ae. aegypti indoor abundance and ABV infection rates monthly during the period of active surveillance. Epidemiological and entomological evaluation will continue for up to three transmission seasons.

DISCUSSION

The findings from this study will provide robust epidemiological evidence of the efficacy of TIRS in reducing ABV illness and infection. If efficacious, TIRS could drive a paradigm shift in Aedes control by considering Ae. aegypti behavior to guide residual insecticide applications and changing deployment to preemptive control (rather than in response to symptomatic cases), two major enhancements to existing practice.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04343521 . Registered on 13 April 2020. The protocol also complies with the WHO International Clinical Trials Registry Platform (ICTRP) (Additional file 1).

PRIMARY SPONSOR

National Institutes of Health, National Institute of Allergy and Infectious Diseases (NIH/NIAID).

Quantifying the localized relationship between vector containment activities and dengue incidence in a real-world setting: A spatial and time series modelling analysis based on geo-located data from Pakistan

Increasing urbanization is having a profound effect on infectious disease risk, posing significant challenges for governments to allocate limited resources for their optimal control at a sub-city scale. With recent advances in data collection practices, empirical evidence about the efficacy of highly localized containment and intervention activities, which can lead to optimal deployment of resources, is possible. However, there are several challenges in analyzing data from such real-world observational settings. Using data on 3.9 million instances of seven dengue vector containment activities collected between 2012 and 2017, here we develop and assess two frameworks for understanding how the generation of new dengue cases changes in space and time with respect to application of different types of containment activities. Accounting for the non-random deployment of each containment activity in relation to dengue cases and other types of containment activities, as well as deployment of activities in different epidemiological contexts, results from both frameworks reinforce existing knowledge about the efficacy of containment activities aimed at the adult phase of the mosquito lifecycle. Results show a 10% (95% CI: 1-19%) and 20% reduction (95% CI: 4-34%) reduction in probability of a case occurring in 50 meters and 30 days of cases which had Indoor Residual Spraying (IRS) and fogging performed in the immediate vicinity, respectively, compared to cases of similar epidemiological context and which had no containment in their vicinity. Simultaneously, limitations due to the real-world nature of activity deployment are used to guide recommendations for future deployment of resources during outbreaks as well as data collection practices. Conclusions from this study will enable more robust and comprehensive analyses of localized containment activities in resource-scarce urban settings and lead to improved allocation of resources of government in an outbreak setting.

The impact of insecticide treated curtains on dengue virus transmission: A cluster randomized trial in Iquitos, Peru

Dengue is one of the most important vector-borne diseases, resulting in an estimated hundreds of millions of infections annually throughout the tropics. Control of dengue is heavily dependent upon control of its primary mosquito vector, Aedes aegypti. Innovative interventions that are effective at targeting the adult stage of the mosquito are needed to increase the options for effective control. The use of insecticide-treated curtains (ITCs) has previously been shown to significantly reduce the abundance of Ae. aegypti in and around homes, but the impact of ITCs on dengue virus (DENV) transmission has not been rigorously quantified. A parallel arm cluster-randomized controlled trial was conducted in Iquitos, Peru to quantify the impact of ITCs on DENV seroconversion as measured through plaque-reduction neutralization tests. Seroconversion data showed that individuals living in the clusters that received ITCs were at greater risk to seroconverting to DENV, with an average seroconversion rate of 50.6 per 100 person-years (PY) (CI: 29.9–71.9), while those in the control arm had an average seroconversion rate of 37.4 per 100 PY (CI: 15.2–51.7). ITCs lost their insecticidal efficacy within 6 months of deployment, necessitating re-treatment with insecticide. Entomological indicators did not show statistically significant differences between ITC and non-ITC clusters. It’s unclear how the lack of protective efficacy reported here is attributable to simple failure of the intervention to protect against Ae. aegypti bites, or the presence of a faulty intervention during much of the follow-up period. The higher risk of dengue seroconversion that was detected in the ITC clusters may have arisen due to a false sense of security that inadvertently led to less routine protective behaviors on the part of households that received the ITCs. Our study provides important lessons learned for conducting cluster randomized trials for vector control interventions against Aedes-transmitted virus infections.

Dengue is one of the most important mosquito-borne diseases affecting humans, resulting in an estimated hundreds of millions of infections annually throughout the tropics. To control dengue, most public health programs use a variety of methods to kill the primary mosquito vector, Aedes aegypti. Water holding containers that harbor larvae (and other immature stages) are treated or eliminated. During emergencies, large insecticide spray campaigns are deployed to kill infected adult mosquitoes. Innovative interventions that are effective at targeting adult mosquitoes in sustainable ways are needed to increase the options for control of dengue and other Aedes borne virus diseases. The use of insecticide-treated curtains (ITCs) has previously been shown to significantly reduce Ae. aegypti numbers in and around homes, but the impact of ITCs on dengue virus (DENV) transmission has not previously been quantified. Using a rigorous study design in which 10 clusters (~90 houses per cluster) were provided multiple ITCs to place in their homes was compared to 10 clusters of homes without ITCs. Assignment of which clusters received ITCs was randomized. Blood samples were obtained at 9-month intervals from residents living in all the clusters, so that people with serological evidence of a DENV infection could be identified by comparing paired samples. Seroconversion data showed that individuals living in the clusters that received ITCs were at greater risk to DENV seroconverting, with an average seroconversion rate of 50.6 per 100 person-years (PY) (CI: 29.9–71.9). Conversely, those in the control arm had an average seroconversion rate of 37.4 per 100 PY (CI: 15.2–51.7). ITCs lost their insecticidal efficacy within 6 months of deployment, necessitating re-treatment with insecticide. Ae. aegypti populations did not show statistically significant differences between ITC and non-ITC clusters. The reason for higher transmission in the ITC treated clusters could be attributable to failure of the curtains (loss of efficacy) and/or that the curtains were not sufficiently effective at protecting against mosquito bites. The higher risk of DENV seroconversion in ITC clusters may be due to a false sense of security that inadvertently led to less routine protective behaviors on the part of households that received the ITC.

Analysis of counts for cluster randomized trials: Negative controls and test-negative designs

In cluster randomized trials (CRTs), the outcome of interest is often a count at the cluster level. This occurs, for example, in evaluating an intervention with the outcome being the number of infections of a disease such as HIV or dengue or the number of hospitalizations in the cluster. Standard practice analyzes these counts through cluster outcome rates using an appropriate denominator (eg, population size). However, such denominators are sometimes unknown, particularly when the counts depend on a passive community surveillance system. We consider direct comparison of the counts without knowledge of denominators, relying on randomization to balance denominators. We also focus on permutation tests to allow for small numbers of randomized clusters. However, such approaches are subject to bias when there is differential ascertainment of counts across arms, a situation that may occur in CRTs that cannot implement blinded interventions. We suggest the use of negative control counts as a method to remove, or reduce, this bias, discussing the key properties necessary for an effective negative control. A current example of such a design is the recent extension of test-negative designs to CRTs testing community-level interventions. Via simulation, we compare the performance of new and standard estimators based on CRTs with negative controls to approaches that only use the original counts. When there is no differential ascertainment by intervention arm, the count-only approaches perform comparably to those using debiasing negative controls. However, under even modest differential ascertainment, the count-only estimators are no longer reliable.

The Use of Test-negative Controls to Monitor Vaccine Effectiveness: A Systematic Review of Methodology

BACKGROUND

The test-negative design is an increasingly popular approach for estimating vaccine effectiveness (VE) due to its efficiency. This review aims to examine published test-negative design studies of VE and to explore similarities and differences in methodological choices for different diseases and vaccines.

METHODS

We conducted a systematic search on PubMed, Web of Science, and Medline, for studies reporting the effectiveness of any vaccines using a test-negative design. We screened titles and abstracts and reviewed full texts to identify relevant articles. We created a standardized form for each included article to extract information on the pathogen of interest, vaccine(s) being evaluated, study setting, clinical case definition, choices of cases and controls, and statistical approaches used to estimate VE.

RESULTS

We identified a total of 348 articles, including studies on VE against influenza virus (n = 253), rotavirus (n = 48), pneumococcus (n = 24), and nine other pathogens. Clinical case definitions used to enroll patients were similar by pathogens of interest but the sets of symptoms that defined them varied substantially. Controls could be those testing negative for the pathogen of interest, those testing positive for nonvaccine type of the pathogen of interest, or a subset of those testing positive for alternative pathogens. Most studies controlled for age, calendar time, and comorbidities.

CONCLUSIONS

Our review highlights similarities and differences in the application of the test-negative design that deserve further examination. If vaccination reduces disease severity in breakthrough infections, particular care must be taken in interpreting vaccine effectiveness estimates from test-negative design studies.

Analysis of cluster-randomized test-negative designs: cluster-level methods

Intervention trials of vector control methods often require community level randomization with appropriate inferential methods. For many interventions, the possibility of confounding due to the effects of health-care seeking behavior on disease ascertainment remains a concern. The test-negative design, a variant of the case-control method, was introduced to mitigate this issue in the assessment of the efficacy of influenza vaccination (measured at an individual level) on influenza infection. Here, we introduce a cluster-randomized test-negative design that includes randomization of the intervention at a group level. We propose several methods for estimation and inference regarding the relative risk (RR). The inferential methods considered are based on the randomization distribution induced by permuting intervention assignment across two sets of randomly selected clusters. The motivating example is a current study of the efficacy of randomized releases of Wolbachia-infected Aedes aegypti mosquitoes to reduce the incidence of dengue in Yogyakarta City, Indonesia. Estimation and inference techniques are assessed through a simulation study.

Measurement of Vaccine Direct Effects Under the Test-Negative Design

Test-negative designs are commonplace in assessments of influenza vaccination effectiveness, estimating this value from the exposure odds ratio of vaccination among individuals treated for acute respiratory illness who test positive for influenza virus infection. This approach is widely believed to recover the vaccine direct effect by correcting for differential health-care-seeking behavior among vaccinated and unvaccinated persons. However, the relationship of the measured odds ratio to true vaccine effectiveness is poorly understood. We derived the odds ratio under circumstances of real-world test-negative studies. The odds ratio recovers the vaccine direct effect when 2 conditions are met: 1) Individuals' vaccination decisions are uncorrelated with exposure or susceptibility to the test-positive or test-negative conditions, and 2) vaccination confers "all-or-nothing" protection (whereby certain individuals have no protection while others are perfectly protected). Biased effect-size estimates arise if either condition is unmet. Such bias might suggest misleading associations of vaccine effectiveness with time since vaccination or the force of infection of influenza. The test-negative design could also fail to correct for differential health-care-seeking behavior among vaccinated and unvaccinated persons without stringent criteria for enrollment and testing. Our findings demonstrate a need to reassess how data from test-negative studies can inform policy decisions.

The AWED trial (Applying Wolbachia to Eliminate Dengue) to assess the efficacy of Wolbachia-infected mosquito deployments to reduce dengue incidence in Yogyakarta, Indonesia: study protocol for a cluster randomised controlled trial

Background

Dengue and other arboviruses transmitted by Aedes aegypti mosquitoes, including Zika and chikungunya, present an increasing public health challenge in tropical regions. Current vector control strategies have failed to curb disease transmission, but continue to be employed despite the absence of robust evidence for their effectiveness or optimal implementation. The World Mosquito Program has developed a novel approach to arbovirus control using Ae. aegypti stably transfected with Wolbachia bacterium, with a significantly reduced ability to transmit dengue, Zika and chikungunya in laboratory experiments. Modelling predicts this will translate to local elimination of dengue in most epidemiological settings. This study protocol describes the first trial to measure the efficacy of Wolbachia in reducing dengue virus transmission in the field.

Methods/design

The study is a parallel, two-arm, non-blinded cluster randomised controlled trial conducted in a single site in Yogyakarta, Indonesia. The aim is to determine whether large-scale deployment of Wolbachia-infected Ae. aegypti mosquitoes leads to a measurable reduction in dengue incidence in treated versus untreated areas. The primary endpoint is symptomatic, virologically confirmed dengue virus infection of any severity. The 26 km² study area was subdivided into 24 contiguous clusters, allocated randomly 1:1 to receive Wolbachia deployments or no intervention. We use a novel epidemiological study design, the cluster-randomised test-negative design trial, in which dengue cases and arbovirus-negative controls are sampled concurrently from among febrile patients presenting to a network of primary care clinics, with case or control status classified retrospectively based on the results of laboratory diagnostic testing. Efficacy is estimated from the odds ratio of Wolbachia exposure distribution (probability of living in a Wolbachia-treated area) among virologically confirmed dengue cases compared to test-negative controls. A secondary per-protocol analysis allows for individual Wolbachia exposure levels to be assessed to account for movements outside the cluster and the heterogeneity in local Wolbachia prevalence among treated clusters.

Discussion

The findings from this study will provide the first experimental evidence for the efficacy of Wolbachia in reducing dengue incidence. Together with observational evidence that is accumulating from pragmatic deployments of Wolbachia in other field sites, this will provide valuable data to estimate the effectiveness of this novel approach to arbovirus control, inform future cost-effectiveness estimates, and guide plans for large-scale deployments in other endemic settings.

Trial registration

ClinicalTrials.gov, identifier: NCT03055585. Registered on 14 February 2017.

Electronic supplementary material

The online version of this article (10.1186/s13063-018-2670-z) contains supplementary material, which is available to authorized users.