Is more research always needed? Estimating optimal sample sizes for trials of retention in care interventions for HIV-positive East Africans

Practical metrics for establishing the health benefits of research to support research prioritisation

INTRODUCTION

We present practical metrics for estimating the expected health benefits of specific research proposals. These can be used by research funders, researchers and healthcare decision-makers within low-income and middle-income countries to support evidence-based research prioritisation.

METHODS

The methods require three key assessments: (1) the current level of uncertainty around the endpoints the proposed study will measure; (2) how uncertainty impacts on the health benefits and costs of healthcare programmes and (3) the health opportunity costs imposed by programme costs. Research is valuable because it can improve health by informing the choice of which programmes should be implemented. We provide a Microsoft Excel tool to allow readers to generate estimates of the health benefits of research studies based on these three assessments. The tool can be populated using existing studies, existing cost-effectiveness models and expert opinion. Where such evidence is not available, the tool can quantify the value of research under different assumptions. Estimates of the health benefits of research can be considered alongside research costs, and the consequences of delaying implementation until research reports, to determine whether research is worthwhile. We illustrate the method using a case study of research on HIV self-testing programmes in Malawi. This analysis combines data from the literature with outputs from the HIV synthesis model.

RESULTS

For this case study, we found a costing study that could be completed and inform decision making within 1 year offered the highest health benefits (67 000 disability-adjusted life years (DALYs) averted). Research on outcomes improved population health to a lesser extent (12 000 DALYs averted) and only if carried out alongside programme implementation.

CONCLUSION

Our work provides a method for estimating the health benefits of research in a practical and timely fashion. This can be used to support accountable use of research funds.

Appraising the value of evidence generation activities: an HIV modelling study

Introduction

The generation of robust evidence has been emphasised as a priority for global health. Evidence generation spans a wide range of activities including clinical trials, surveillance programmes and health system performance measurement. As resources for healthcare and research are limited, the desirability of research expenditure should be assessed on the same basis as other healthcare resources, that is, the health gains from research must be expected to exceed the health opportunity costs imposed as funds are diverted to research rather than service provision.

Methods

We developed a transmission and costing model to examine the impact of generating additional evidence to reduce uncertainties on the evolution of a generalised HIV epidemic in Zambia.

Results

We demonstrate three important points. First, we can quantify the value of additional evidence in terms of the health gain it is expected to generate. Second, we can quantify the health opportunity cost imposed by research expenditure. Third, the value of evidence generation depends on the budgetary policies in place for managing HIV resources under uncertainty. Generating evidence to reduce uncertainty is particularly valuable when decision makers are required to strictly adhere to expenditure plans and when transfers of funds across geographies/programmes are restricted.

Conclusion

Better evidence can lead to health improvements in the same way as direct delivery of healthcare. Quantitative appraisals of evidence generation activities are important and should reflect the impact of improved evidence on population health, evidence generation costs and budgetary policies in place.

Using value of information methods to determine the optimal sample size for effectiveness trials of alcohol interventions for HIV-infected patients in East Africa

Background

Unhealthy alcohol consumption exacerbates the HIV epidemic in East Africa. Potential benefits of new trials that test the effectiveness of alcohol interventions could not be evaluated by traditional sampling methods. Given the competition for health care resources in East Africa, this study aims to determine the optimal sample size given the opportunity cost of potentially re-allocating trial funds towards cost-effective alcohol treatments.

Methods

We used value of information methods to determine the optimal sample size by maximizing the expected net benefit of sampling for a hypothetical 2-arm intervention vs. control randomized trial, across ranges of policymaker’s willingness-to-pay for the health benefit of an intervention. Probability distributions describing the relative likelihood of alternative trial results were imputed based on prior studies. In the base case, policymaker’s willingness-to-pay was based on a simultaneously resource-constrained priority (routine HIV virological testing). Sensitivity analysis was performed for various willingness-to-pay thresholds and intervention durations.

Results

A new effectiveness trial accounting for the benefit of more precise decision-making on alcohol intervention implementation would benefit East Africa $67,000 with the optimal sample size of 100 persons per arm under the base case willingness-to-pay threshold and intervention duration of 20 years. At both a conservative willingness-to-pay of 1 x GDP/capita and a high willingness-to-pay of 3 x GDP/capita for an additional health gain added by an alcohol intervention, a new trial was not recommended due to limited decision uncertainty. When intervention duration was 10 or 5 years, there was no return on investment across suggested willingness-to-pay thresholds.

Conclusions

Value of information methods could be used as an alternative approach to assist the efficient design of alcohol trials. If reducing unhealthy alcohol use is a long-term goal for HIV programs in East Africa, additional new trials with optimal sample sizes ranging from 100 to 250 persons per arm could save the opportunity cost of implementing less cost-effective alcohol strategies in HIV prevention. Otherwise, conducting a new trial is not recommended.