Appraising the value of evidence generation activities: an HIV modelling study

Academic Clinical Trials Submitted to an Institutional Ethics Committee at a Tertiary Care Center: A Retrospective Analysis

Background The role of academia in clinical research has given rise to the concept of academic clinical trials (ACTs), which are vital in generating evidence. Through the implementation of the New Drugs and Clinical Trials Rules-2019 (NDCTR-2019) rules, the Institutional Ethics Committee (IEC) has obtained a quasi-regulatory role. The study aims to assess the challenges the IEC faced when processing, approving, and monitoring ACTs. The other objectives included the number of ACTs submitted to the IEC, as well as administrative, scientific, and ethical issues stated by the IEC and the Drug Controller General of India (DCGI) authorities. We also aimed to provide some insight into the type of decision made by IEC and DCGI - the delay or inconsistency between the queries. Methods This retrospective study was conducted in the IEC of a tertiary care hospital, Mumbai, Maharashtra, India. A comprehensive search of the IEC database was carried out by the study team, and only those protocols of ACTs submitted to IEC between January 2015 and December 2021 were included. The studies submitted between January 2015 and February 2019, i.e., before the release of NDCTR-2019, were classified as the "Before" category. All subsequently submitted protocols were grouped together as the "After" group. Descriptive statistics were used to represent the data, while comparison between the two timeframes were made using the Mann-Whitney U test with a level of significance at 5%. Results This six-year study showed that merely 1.4% (34/2400) trials fulfilled the criteria of an ACT. An increase in the ACT protocol submission was noted in the "After" group (20 vs. 14). Most ACTs were drug trials, with 67.6% (23/34) trials conducted majorly in the Department of Anesthesiology. There was a statistical increase in time query reply by the principal investigator to IEC and the time between submission and approval in the "After" group (p<0.05). IEC sent out 94 administrative, 565 scientific, and 216 ethical queries. On IEC monitoring, protocol deviations were noted; nonetheless, no ACTs reported protocol deviations or serious adverse events. Conclusions Since the implementation of NDCTR-2019, IEC has taken on a quasi-regulatory function, and there has been an increase in the caliber of IEC monitoring and adherence to ethical norms.

Optimal Epidemic Control under Uncertainty: Tradeoffs between Information Collection and Other Actions

BACKGROUND

Recent epidemics and measures taken to control them-through vaccination or other actions-have highlighted the role and importance of uncertainty in public health. There is generally a tradeoff between information collection and other uses of resources. Whether this tradeoff is solved explicitly or implicitly, the concept of value of information is central to inform policy makers in an uncertain environment.

METHOD

We use a deterministic SIR (susceptible, infectious, recovered) disease emergence and transmission model with vaccination that can be administered as 1 or 2 doses. The disease parameters and vaccine characteristics are uncertain. We study the tradeoffs between information acquisition and 2 other measures: bringing vaccination forward and acquiring more vaccine doses. To do this, we quantify the expected value of perfect information (EVPI) under different constraints faced by public health authorities (i.e., the time of the vaccination campaign implementation and the number of vaccine doses available).

RESULTS

We discuss the appropriateness of different responses under uncertainty. We show that, in some cases, vaccinating later or with less vaccine doses but more information about the epidemic, and the efficacy of control strategies may bring better results than vaccinating earlier or with more doses and less information, respectively.

CONCLUSION

In the present methodological article, we show in an abstract setting how clearly defining and treating the tradeoff between information acquisition and the relaxation of constraints can improve public health decision making.

HIGHLIGHTS

Uncertainties can seriously hinder epidemic control, but resolving them is costly. Thus, there are tradeoffs between information collection and alternative uses of resources.We use a generic SIR model with vaccination and a value-of-information framework to explore these tradeoffs.We show in which cases vaccinating later with more information about the epidemic and the efficacy of control measures may be better-or not-than vaccinating earlier with less information.We show in which cases vaccinating with fewer vaccine doses and more information about the epidemic and the efficacy of control measures may be better-or not-than vaccinating with more doses and less information.

Bayesian validation framework for dynamic epidemic models

Complex models of infectious diseases are used to understand the transmission dynamics of the disease, project the course of an epidemic, predict the effect of interventions and/or provide information for power calculations of community level intervention studies. However, there have been relatively few opportunities to rigorously evaluate the predictions of such models till now. Indeed, while there is a large literature on calibration (fitting model parameters) and validation (comparing model outputs to data) of complex models based on empirical data, the lack of uniformity in accepted criteria for such procedures for models of infectious diseases has led to simple procedures being prevalent for such steps. However, recently, several community level randomized trials of combination HIV intervention have been planned and/or initiated, and in each case, significant epidemic modeling efforts were conducted during trial planning which were integral to the design of these trials. The existence of these models and the (anticipated) availability of results from the related trials, provide a unique opportunity to evaluate the models and their usefulness in trial design. In this project, we outline a framework for evaluating the predictions of complex epidemiological models and describe experiments that can be used to test their predictions.

Practical metrics for establishing the health benefits of research to support research prioritisation

INTRODUCTION

We present practical metrics for estimating the expected health benefits of specific research proposals. These can be used by research funders, researchers and healthcare decision-makers within low-income and middle-income countries to support evidence-based research prioritisation.

METHODS

The methods require three key assessments: (1) the current level of uncertainty around the endpoints the proposed study will measure; (2) how uncertainty impacts on the health benefits and costs of healthcare programmes and (3) the health opportunity costs imposed by programme costs. Research is valuable because it can improve health by informing the choice of which programmes should be implemented. We provide a Microsoft Excel tool to allow readers to generate estimates of the health benefits of research studies based on these three assessments. The tool can be populated using existing studies, existing cost-effectiveness models and expert opinion. Where such evidence is not available, the tool can quantify the value of research under different assumptions. Estimates of the health benefits of research can be considered alongside research costs, and the consequences of delaying implementation until research reports, to determine whether research is worthwhile. We illustrate the method using a case study of research on HIV self-testing programmes in Malawi. This analysis combines data from the literature with outputs from the HIV synthesis model.

RESULTS

For this case study, we found a costing study that could be completed and inform decision making within 1 year offered the highest health benefits (67 000 disability-adjusted life years (DALYs) averted). Research on outcomes improved population health to a lesser extent (12 000 DALYs averted) and only if carried out alongside programme implementation.

CONCLUSION

Our work provides a method for estimating the health benefits of research in a practical and timely fashion. This can be used to support accountable use of research funds.

The importance of local epidemic conditions in monitoring progress towards HIV epidemic control in Kenya: a modelling study

INTRODUCTION

Setting and monitoring progress towards targets for HIV control is critical in ensuring responsive programmes. Here, we explore how to apply targets for reduction in HIV incidence to local settings and which indicators give the strongest signal of a change in incidence in the population and are therefore most important to monitor.

METHODS

We use location-specific HIV transmission models, tailored to the epidemics in the counties and major cities in Kenya, to project a wide range of plausible future epidemic trajectories through varying behaviours, treatment coverage and prevention interventions. We look at the change in incidence across modelled scenarios in each location between 2015 and 2030 to inform local target setting. We also simulate the measurement of a library of potential indicators and assess which are most strongly associated with a change in incidence.

RESULTS

Considerable variation was observed in the trajectory of the local epidemics under the plausible scenarios defined (only 10 of 48 locations saw a median reduction in incidence of greater than or equal to an 80% target by 2030). Indicators that provide strong signals in certain epidemic types may not perform consistently well in settings with different epidemiological features. Predicting changes in incidence is more challenging in advanced generalized epidemics compared to concentrated epidemics where changes in high-risk sub-populations track more closely to the population as a whole. Many indicators demonstrate only limited association with incidence (such as "condom use" or "pre-exposure prophylaxis coverage"). This is because many other factors (low effectiveness, impact of other interventions, countervailing changes in risk behaviours, etc.) can confound the relationship between interventions and their ultimate long-term impact, especially for an intervention with low expected coverage. The population prevalence of viral suppression shows the most consistent associations with long-term changes in incidence even in the largest generalized epidemics.

CONCLUSIONS

Target setting should be appropriate for the local epidemic and what can feasibly be achieved. There is no one universally reliable indicator to predict future HIV incidence across settings. Thus, the signature of epidemic control must contain indications of success across a wide range of interventions and outcomes.