Using simulation to aid trial design: Ring-vaccination trials

Value of information dynamics in Disease X vaccine clinical trials

BACKGROUND

Solutions have been proposed to accelerate the development and rollout of vaccines against a hypothetical disease with epidemic or pandemic potential called Disease X. This may involve resolving uncertainties regarding the disease and the new vaccine. However the value for public health of collecting this information will depend on the time needed to perform research, but also on the time needed to produce vaccine doses. We explore this interplay, and its effect on the decision on whether or not to perform research.

METHOD

We simulate numerically the emergence and transmission of a disease in a population using a susceptible-infected-recovered (SIR) compartmental model with vaccination. Uncertainties regarding the disease and the vaccine are represented by parameter prior distributions. We vary the date at which vaccine doses are available, and the date at which information about parameters becomes available. We use the expected value of perfect information (EVPI) and the expected value of partially perfect information (EVPPI) to measure the value of information.

RESULTS

As expected, information has less or no value if it comes too late, or (equivalently) if it can only be used too late. However we also find non trivial dynamics for shorter durations of vaccine development. In this parameter area, it can be optimal to implement vaccination without waiting for information depending on the respective durations of dose production and of clinical research.

CONCLUSION

We illustrate the value of information dynamics in a Disease X outbreak scenario, and present a general approach to properly take into account uncertainties and transmission dynamics when planning clinical research in this scenario. Our method is based on numerical simulation and allows us to highlight non trivial effects that cannot otherwise be investigated.

Expected endpoints from future chikungunya vaccine trial sites informed by serological data and modeling

In recent decades, there has been an increased interest in developing a vaccine for chikungunya. However, due to its unpredictable transmission, planning for a chikungunya vaccine trial is challenging. To inform decision making on the selection of sites for a vaccine efficacy trial, we developed a new framework for projecting the expected number of endpoint events at a given site. In this framework, we first accounted for population immunity using serological data collated from a systematic review and used it to estimate parameters related to the timing and size of past outbreaks, as predicted by an SIR transmission model. Then, we used that model to project the infection attack rate of a hypothetical future outbreak, in the event that one were to occur at the time of a future trial. This informed projections of how many endpoint events could be expected if a trial were to take place at that site. Our results suggest that some sites may have sufficient transmission potential and susceptibility to support future vaccine trials, in the event that an outbreak were to occur at those sites. In general, we conclude that sites that have experienced outbreaks within the past 10 years may be poorer targets for chikungunya vaccine efficacy trials in the near future. Our framework also generates projections of the numbers of endpoint events by age, which could inform study participant recruitment efforts.

A Review of the Ring Trial Design for Evaluating Ring Interventions for Infectious Diseases

In trials of infectious disease interventions, rare outcomes and unpredictable spatiotemporal variation can introduce bias, reduce statistical power, and prevent conclusive inferences. Spillover effects can complicate inference if individual randomization is used to gain efficiency. Ring trials are a type of cluster-randomized trial that may increase efficiency and minimize bias, particularly in emergency and elimination settings with strong clustering of infection. They can be used to evaluate ring interventions, which are delivered to individuals in proximity to or contact with index cases. We conducted a systematic review of ring trials, compare them with other trial designs for evaluating ring interventions, and describe strengths and weaknesses of each design. Of 849 articles and 322 protocols screened, we identified 26 ring trials, 15 cluster-randomized trials, 5 trials that randomized households or individuals within rings, and 1 individually randomized trial. The most common interventions were postexposure prophylaxis (n = 23) and focal mass drug administration and screening and treatment (n = 7). Ring trials require robust surveillance systems and contact tracing for directly transmitted diseases. For rare diseases with strong spatiotemporal clustering, they may have higher efficiency and internal validity than cluster-randomized designs, in part because they ensure that no clusters are excluded from analysis due to zero cluster incidence. Though more research is needed to compare them with other types of trials, ring trials hold promise as a design that can increase trial speed and efficiency while reducing bias.

Chlamydia trachomatis vaccine development - a view on the current challenges and how to move forward

INTRODUCTION

Chlamydia trachomatis is the most common sexually transmitted bacterial pathogen in the world. A licensed vaccine is not yet available, but the first vaccines have entered clinical trials.

AREAS COVERED

: We describe the progress that has been made in our understanding of the type of immunity that a protective vaccine should induce, and the challenges that vaccine developers face. We also focus on the clinical development of a chlamydia vaccine. The first chlamydia vaccine candidate has now been tested in a clinical phase-I trial, and another phase-I trial is currently running. We discuss what it will take to continue this development and what future trial setups could look like.

EXPERT OPINION

The chlamydia field is coming of age and the first phase I clinical trial of a C. trachomatis vaccine has been successfully completed. We expect and hope that this will motivate various stakeholders to support further development of chlamydia vaccines in humans.

The ring vaccination trial design for the estimation of vaccine efficacy and effectiveness during infectious disease outbreaks

The ring vaccination trial is a recently developed approach for evaluating the efficacy and effectiveness of vaccines, modeled after the surveillance and containment strategy of ring vaccination. Contacts and contacts of contacts of a newly identified disease case form a ring, and these rings are randomized as part of a cluster-randomized trial or with individual randomization within rings. Key advantages of the design include its flexibility to follow the epidemic as it progresses and the targeting of high-risk participants to increase power. We describe the application of the design to estimate the efficacy and effectiveness of an Ebola vaccine during the 2014-2016 West African Ebola epidemic. The design has several notable statistical features. Because vaccination occurs around the time of exposure, the design is particularly sensitive to the choice of per protocol analysis period. If incidence wanes before the per protocol analysis period begins (due to a slow-acting vaccine or a fast-moving pathogen), power can be substantially reduced. Mathematical modeling is valuable for exploring the suitability of the approach in different disease settings. Another statistical feature is zero inflation, which can occur if the chain of transmission does not take off within a ring. In the application to Ebola, the majority of rings had zero subsequent cases. The ring vaccination trial can be extended in several ways, including the definition of rings (e.g. contact-based, spatial, and occupational). The design will be valuable in settings where the spatio-temporal spread of the pathogen is highly focused and unpredictable.

Response adaptive intervention allocation in stepped-wedge cluster randomized trials

BACKGROUND

Stepped-wedge cluster randomized trial (SW-CRT) designs are often used when there is a desire to provide an intervention to all enrolled clusters, because of a belief that it will be effective. However, given there should be equipoise at trial commencement, there has been discussion around whether a pre-trial decision to provide the intervention to all clusters is appropriate. In pharmaceutical drug development, a solution to a similar desire to provide more patients with an effective treatment is to use a response adaptive (RA) design.

METHODS

We introduce a way in which RA design could be incorporated in an SW-CRT, permitting modification of the intervention allocation during the trial. The proposed framework explicitly permits a balance to be sought between power and patient benefit considerations. A simulation study evaluates the methodology.

RESULTS

In one scenario, for one particular RA design, the proportion of cluster-periods spent in the intervention condition was observed to increase from 32.2% to 67.9% as the intervention effect was increased. A cost of this was a 6.2% power drop compared to a design that maximized power by fixing the proportion of time in the intervention condition at 45.0%, regardless of the intervention effect.

CONCLUSIONS

An RA approach may be most applicable to settings for which the intervention has substantial individual or societal benefit considerations, potentially in combination with notable safety concerns. In such a setting, the proposed methodology may routinely provide the desired adaptability of the roll-out speed, with only a small cost to the study's power.

Inference is bliss: Simulation for power estimation for an observational study of a cholera outbreak intervention

Background

The evaluation of ring vaccination and other outbreak-containment interventions during severe and rapidly-evolving epidemics presents a challenge for the choice of a feasible study design, and subsequently, for the estimation of statistical power. To support a future evaluation of a case-area targeted intervention against cholera, we have proposed a prospective observational study design to estimate the association between the strength of implementation of this intervention across several small outbreaks (occurring within geographically delineated clusters around primary and secondary cases named ‘rings’) and its effectiveness (defined as a reduction in cholera incidence). We describe here a strategy combining mathematical modelling and simulation to estimate power for a prospective observational study.

Methodology and principal findings

The strategy combines stochastic modelling of transmission and the direct and indirect effects of the intervention in a set of rings, with a simulation of the study analysis on the model results. We found that targeting 80 to 100 rings was required to achieve power ≥80%, using a basic reproduction number of 2.0 and a dispersion coefficient of 1.0–1.5.

Conclusions

This power estimation strategy is feasible to implement for observational study designs which aim to evaluate outbreak containment for other pathogens in geographically or socially defined rings.

From Ebola virus disease outbreaks to the COVID-19 pandemic, the use of real-time evaluations of interventions to contain outbreaks is vital for rapidly estimating impact during the outbreak itself. Such evaluations must be both epidemiologically rigorous and logistically feasible to justify their conduct during an outbreak. In this short report, we report on the process (with R code) and the results of a simulation strategy that we devised for power estimation for a prospective observational study of a novel intervention (“case-area targeted intervention”) to contain cholera case clusters that present at the start of a new outbreak. We used simulation in two ways: mathematical modelling to simulate the impacts of a cholera outbreak and the intervention, and simulation of the study analysis on the model results. The strategy provided estimates of the sample sizes of study units required to achieve 80% and 90% power. Our findings reinforce that this process is feasible to implement for similar observational study designs which aim to evaluate outbreak containment for other pathogens in geographically or socially defined rings.

Quantifying Efficiency Gains of Innovative Designs of Two-Arm Vaccine Trials for COVID-19 Using an Epidemic Simulation Model

Clinical trials of a vaccine during an epidemic face particular challenges, such as the pressure to identify an effective vaccine quickly to control the epidemic, and the effect that time-space-varying infection incidence has on the power of a trial. We illustrate how the operating characteristics of different trial design elements maybe evaluated using a network epidemic and trial simulation model, based on COVID-19 and individually randomized two-arm trials with a binary outcome. We show that "ring" recruitment strategies, prioritizing participants at an imminent risk of infection, can result in substantial improvement in terms of power in the model we present. In addition, we introduce a novel method to make more efficient use of the data from the earliest cases of infection observed in the trial, whose infection may have been too early to be vaccine-preventable. Finally, we compare several methods of response-adaptive randomization (RAR), discussing their advantages and disadvantages in the context of our model and identifying particular adaptation strategies that preserve power and estimation properties, while slightly reducing the number of infections, given an effective vaccine.

Assessing the feasibility of Nipah vaccine efficacy trials based on previous outbreaks in Bangladesh

BACKGROUND

Nipah virus (NiV) is an emerging, bat-borne pathogen that can be transmitted from person-to-person. Vaccines are currently being developed for NiV, and studies have been funded to evaluate their safety and immunogenicity. An important unanswered question is whether it will be possible to evaluate the efficacy of vaccine candidates in phase III clinical trials in a context where spillovers from the zoonotic reservoir are infrequent and associated with small outbreaks. The objective of this study was to investigate the feasibility of conducting a phase III vaccine trial in Bangladesh, the only country regularly reporting NiV cases.

METHODS

We used simulations based on previously observed NiV cases from Bangladesh, an assumed vaccine efficacy of 90% and other NiV vaccine target characteristics, to compare three vaccination study designs: (i) cluster randomized ring vaccination, (ii) cluster randomized mass vaccination, and (iii) an observational case-control study design.

RESULTS

The simulations showed that, assuming a ramp-up period of 10 days and a mean hospitalization delay of 4 days,a cluster-randomized ring vaccination trial would require 516 years and over 163,000 vaccine doses to run a ring vaccination trial under current epidemic conditions. A cluster-randomized mass vaccination trial in the two most affected districts would take 43 years and 1.83 million vaccine doses. An observational case-control design in these two districts would require seven years and 2.5 million vaccine doses.

DISCUSSION

Without a change in the epidemiology of NiV, ring vaccination or mass vaccination trials are unlikely to be completed within a reasonable time window. In this light, the remaining options are: (i) not conducting a phase III trial until the epidemiology of NiV changes, (ii) identifying alternative ways to licensure such as observational studies or controlled studies in animals such as in the US Food and Drug Administration's (FDA) Animal Rule.

Emulating Target Trials to Improve Causal Inference From Agent-Based Models

Agent-based models are a key tool for investigating the emergent properties of population health settings, such as infectious disease transmission, where the exposure often violates the key "no interference" assumption of traditional causal inference under the potential outcomes framework. Agent-based models and other simulation-based modeling approaches have generally been viewed as a separate knowledge-generating paradigm from the potential outcomes framework, but this can lead to confusion about how to interpret the results of these models in real-world settings. By explicitly incorporating the target trial framework into the development of an agent-based or other simulation model, we can clarify the causal parameters of interest, as well as make explicit the assumptions required for valid causal effect estimation within or between populations. In this paper, we describe the use of the target trial framework for designing agent-based models when the goal is estimation of causal effects in the presence of interference, or spillover.

Using simulated infectious disease outbreaks to inform site selection and sample size for individually randomized vaccine trials during an ongoing epidemic

Background:

Novel strategies are needed to make vaccine efficacy trials more robust given uncertain epidemiology of infectious disease outbreaks, such as arboviruses like Zika. Spatially resolved mathematical and statistical models can help investigators identify sites at highest risk of future transmission and prioritize these for inclusion in trials. Models can also characterize uncertainty in whether transmission will occur at a site, and how nearby or connected sites may have correlated outcomes. A structure is needed for how trials can use models to address key design questions, including how to prioritize sites, the optimal number of sites, and how to allocate participants across sites.

Methods:

We illustrate the added value of models using the motivating example of Zika vaccine trial planning during the 2015–2017 Zika epidemic. We used a stochastic, spatially resolved, transmission model (the Global Epidemic and Mobility model) to simulate epidemics and site-level incidence at 100 high-risk sites in the Americas. We considered several strategies for prioritizing sites (average site-level incidence of infection across epidemics, median incidence, probability of exceeding 1% incidence), selecting the number of sites, and allocating sample size across sites (equal enrollment, proportional to average incidence, proportional to rank). To evaluate each design, we stochastically simulated trials in each hypothetical epidemic by drawing observed cases from site-level incidence data.

Results:

When constraining overall trial size, the optimal number of sites represents a balance between prioritizing highest-risk sites and having enough sites to reduce the chance of observing too few endpoints. The optimal number of sites remained roughly constant regardless of the targeted number of events, although it is necessary to increase the sample size to achieve the desired power. Though different ranking strategies returned different site orders, they performed similarly with respect to trial power. Instead of enrolling participants equally from each site, investigators can allocate participants proportional to projected incidence, though this did not provide an advantage in our example because the top sites had similar risk profiles. Sites from the same geographic region may have similar outcomes, so optimal combinations of sites may be geographically dispersed, even when these are not the highest ranked sites.

Conclusion:

Mathematical and statistical models may assist in designing successful vaccination trials by capturing uncertainty and correlation in future transmission. Although many factors affect site selection, such as logistical feasibility, models can help investigators optimize site selection and the number and size of participating sites. Although our study focused on trial design for an emerging arbovirus, a similar approach can be made for any infectious disease with the appropriate model for the particular disease.

Operational research: A multidisciplinary approach for the management of infectious disease in a global context

Infectious diseases, both established and emerging, impose a significant burden globally. Successful management of infectious diseases requires considerable effort and a multidisciplinary approach to tackle the complex web of interconnected biological, public health and economic systems. Through a wide range of problem-solving techniques and computational methods, operational research can strengthen health systems and support decision-making at all levels of disease control. From improved understanding of disease biology, intervention planning and implementation, assessing economic feasibility of new strategies, identifying opportunities for cost reductions in routine processes, and informing health policy, this paper highlights areas of opportunity for operational research to contribute to effective and efficient infectious disease management and improved health outcomes.

What defines an efficacious COVID-19 vaccine? A review of the challenges assessing the clinical efficacy of vaccines against SARS-CoV-2

The novel coronavirus, severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), has caused more than 1 million deaths in the first 6 months of the pandemic and huge economic and social upheaval internationally. An efficacious vaccine is essential to prevent further morbidity and mortality. Although some countries might deploy COVID-19 vaccines on the strength of safety and immunogenicity data alone, the goal of vaccine development is to gain direct evidence of vaccine efficacy in protecting humans against SARS-CoV-2 infection and COVID-19 so that manufacture of efficacious vaccines can be selectively upscaled. A candidate vaccine against SARS-CoV-2 might act against infection, disease, or transmission, and a vaccine capable of reducing any of these elements could contribute to disease control. However, the most important efficacy endpoint, protection against severe disease and death, is difficult to assess in phase 3 clinical trials. In this Review, we explore the challenges in assessing the efficacy of candidate SARS-CoV-2 vaccines, discuss the caveats needed to interpret reported efficacy endpoints, and provide insight into answering the seemingly simple question, "Does this COVID-19 vaccine work?"

Use of reliable contraceptives and its correlates among women participating in Simulated HIV vaccine efficacy trials in key-populations in Uganda

To prevent pregnancy in trials, reliable contraceptive use is key. We investigated reliable contraceptive use at baseline and six months in key-populations in Uganda, during two Simulated HIV Vaccine Efficacy trials (SiVETs). SiVETs were nested within observational cohorts of Fisherfolk (2012–2014) and Female sex workers (2014–2017). Women in the observational cohorts were screened and enrolled into the SiVET. The trial administered a licensed Hepatitis B vaccine at 0, 1 and 6 months. Contraceptive use data were recorded at baseline and follow-up clinic visits. Reliable contraceptives (injectable Depot Medroxyprogesterone Acetate (DMPA), implant, pills, and intrauterine device (IUD)) were promoted and provided to women not using a reliable method at enrolment. Overall, 367 women were enrolled. At baseline 203 (55%) reported use of reliable contraceptive. Of the 164 women not using a reliable method at enrolment, 131 (80%) started using them during follow-up bringing the overall number to 334 (91%) at the end of follow-up. Young age (≤35 years) was an independent predictor of reliable contraceptive use at both time points while other factors varied. Promotion and provision of reliable contraceptives increased the proportion using them and could help reduce the risk of pregnancy in future HIV prevention trials.

Simulated vaccine efficacy trials to estimate HIV incidence for actual vaccine clinical trials in key populations in Uganda

BACKGROUND

Fisherfolks (FF) and female sex workers (FSW) in Uganda could be suitable key populations for HIV vaccine efficacy trials because of the high HIV incidence and good retention in observational cohorts. However, the observed HIV incidence may differ in participants who enroll into a trial. We used simulated vaccine efficacy trials (SiVET) nested within observational cohorts in these populations to evaluate this difference.

METHODS

SiVETs were nested in two observational cohorts (Jul 2012-Apr 2014 in FF and Aug 2014-Apr 2017 in FSW). From Jan 2012 all observational cohort participants (aged 18-49 years) presenting for quarterly visits were screened for enrolment into SiVETs, until 572 were enrolled. Those not enrolled (screened-out or not screened) in SiVET continued participation in the observational cohorts. In addition to procedures in the observational cohorts (HIV testing & risk assessment), SiVET participants were given a licensed Hepatitis B vaccine mimicking a schedule of a possible HIV vaccine, and followed-up for 12 months.

FINDINGS

In total, 3989 participants were enrolled into observational cohorts (1575 FF prior to Jul 2012 and 2414 FSW prior to Aug 2014). Of these 3622 (90.8%) returned at least once, 672 (44.1%) were screened and 572 enrolled in the SiVETs. HIV incidence pre SIVETs was 4.5/100 person years-at-risk (pyar), 95%CI (3.8-5.5). HIV incidence in SiVET was 3.5/100 pyar, (2.2-5.6) and higher in those not enrolled in the SiVET, 5.9/100 pyar, (4.3-8.1). This difference was greatest among FF. In the 12 months post-SIVET period (FF, May 2014-Apr 2015 and FSW, May 2017-Apr 2018), the HIV incidence was 3.7/100 pyar, (2.5-5.8).

INTERPRETATION

HIV incidence was lower in SiVET participants compared to non-SiVET. This difference was different for the two populations. Researchers designing HIV efficacy trials using observational cohort data need to consider the potential for lower than expected HIV incidence following screening and enrolment.

Choices in vaccine trial design in epidemics of emerging infections

In a Policy Forum, Marc Lipsitch and colleagues discuss trial design issues in infectious disease outbreaks.

The potential impact of case-area targeted interventions in response to cholera outbreaks: A modeling study

BACKGROUND

Cholera prevention and control interventions targeted to neighbors of cholera cases (case-area targeted interventions [CATIs]), including improved water, sanitation, and hygiene, oral cholera vaccine (OCV), and prophylactic antibiotics, may be able to efficiently avert cholera cases and deaths while saving scarce resources during epidemics. Efforts to quickly target interventions to neighbors of cases have been made in recent outbreaks, but little empirical evidence related to the effectiveness, efficiency, or ideal design of this approach exists. Here, we aim to provide practical guidance on how CATIs might be used by exploring key determinants of intervention impact, including the mix of interventions, "ring" size, and timing, in simulated cholera epidemics fit to data from an urban cholera epidemic in Africa.

METHODS AND FINDINGS

We developed a micro-simulation model and calibrated it to both the epidemic curve and the small-scale spatiotemporal clustering pattern of case households from a large 2011 cholera outbreak in N'Djamena, Chad (4,352 reported cases over 232 days), and explored the potential impact of CATIs in simulated epidemics. CATIs were implemented with realistic logistical delays after cases presented for care using different combinations of prophylactic antibiotics, OCV, and/or point-of-use water treatment (POUWT) starting at different points during the epidemics and targeting rings of various radii around incident case households. Our findings suggest that CATIs shorten the duration of epidemics and are more resource-efficient than mass campaigns. OCV was predicted to be the most effective single intervention, followed by POUWT and antibiotics. CATIs with OCV started early in an epidemic focusing on a 100-m radius around case households were estimated to shorten epidemics by 68% (IQR 62% to 72%), with an 81% (IQR 69% to 87%) reduction in cases compared to uncontrolled epidemics. These same targeted interventions with OCV led to a 44-fold (IQR 27 to 78) reduction in the number of people needed to target to avert a single case of cholera, compared to mass campaigns in high-cholera-risk neighborhoods. The optimal radius to target around incident case households differed by intervention type, with antibiotics having an optimal radius of 30 m to 45 m compared to 70 m to 100 m for OCV and POUWT. Adding POUWT or antibiotics to OCV provided only marginal impact and efficiency improvements. Starting CATIs early in an epidemic with OCV and POUWT targeting those within 100 m of an incident case household reduced epidemic durations by 70% (IQR 65% to 75%) and the number of cases by 82% (IQR 71% to 88%) compared to uncontrolled epidemics. CATIs used late in epidemics, even after the peak, were estimated to avert relatively few cases but substantially reduced the number of epidemic days (e.g., by 28% [IQR 15% to 45%] for OCV in a 100-m radius). While this study is based on a rigorous, data-driven approach, the relatively high uncertainty about the ways in which POUWT and antibiotic interventions reduce cholera risk, as well as the heterogeneity in outbreak dynamics from place to place, limits the precision and generalizability of our quantitative estimates.

CONCLUSIONS

In this study, we found that CATIs using OCV, antibiotics, and water treatment interventions at an appropriate radius around cases could be an effective and efficient way to fight cholera epidemics. They can provide a complementary and efficient approach to mass intervention campaigns and may prove particularly useful during the initial phase of an outbreak, when there are few cases and few available resources, or in order to shorten the often protracted tails of cholera epidemics.

Simulations for designing and interpreting intervention trials in infectious diseases

BACKGROUND

Interventions in infectious diseases can have both direct effects on individuals who receive the intervention as well as indirect effects in the population. In addition, intervention combinations can have complex interactions at the population level, which are often difficult to adequately assess with standard study designs and analytical methods.

DISCUSSION

Herein, we urge the adoption of a new paradigm for the design and interpretation of intervention trials in infectious diseases, particularly with regard to emerging infectious diseases, one that more accurately reflects the dynamics of the transmission process. In an increasingly complex world, simulations can explicitly represent transmission dynamics, which are critical for proper trial design and interpretation. Certain ethical aspects of a trial can also be quantified using simulations. Further, after a trial has been conducted, simulations can be used to explore the possible explanations for the observed effects.

CONCLUSION

Much is to be gained through a multidisciplinary approach that builds collaborations among experts in infectious disease dynamics, epidemiology, statistical science, economics, simulation methods, and the conduct of clinical trials.

Improving vaccine trials in infectious disease emergencies

Unprecedented global effort is under way to facilitate the testing of countermeasures in infectious disease emergencies. Better understanding of the various options for trial design is needed in advance of outbreaks, as is preliminary global agreement on the most suitable designs for the various scenarios. What would enhance the speed, validity, and ethics of clinical studies of such countermeasures? Focusing on studies of vaccine efficacy and effectiveness in emergencies, we highlight three needs: for formal randomized trials-even in most emergencies; for individually randomized trials-even in many emergencies; and for six areas of innovation in trial methodology. These needs should inform current updates of protocols and roadmaps.