Improving the quality of modelling evidence used for tuberculosis policy evaluation

Improving the contribution of mathematical modelling evidence to guidelines and policy: Experiences from tuberculosis

We read with great interest the recent paper by Lo et al., who argue that there is an urgent need to ensure the quality of modelling evidence used to support international and national guideline development. Here we outline efforts by the Tuberculosis Modelling and Analysis Consortium, together with the World Health Organization Global Task Force on Tuberculosis Impact Measurement, to develop material to improve the quality and transparency of country-level tuberculosis modelling to inform decision-making.

Tabby2: a user-friendly web tool for forecasting state-level TB outcomes in the United States

BACKGROUND

In the United States, the tuberculosis (TB) disease burden and associated factors vary substantially across states. While public health agencies must choose how to deploy resources to combat TB and latent tuberculosis infection (LTBI), state-level modeling analyses to inform policy decisions have not been widely available.

METHODS

We developed a mathematical model of TB epidemiology linked to a web-based user interface - Tabby2. The model is calibrated to epidemiological and demographic data for the United States, each U.S. state, and the District of Columbia. Users can simulate pre-defined scenarios describing approaches to TB prevention and treatment or create their own intervention scenarios. Location-specific results for epidemiological outcomes, service utilization, costs, and cost-effectiveness are reported as downloadable tables and customizable visualizations. To demonstrate the tool's functionality, we projected trends in TB outcomes without additional intervention for all 50 states and the District of Columbia. We further undertook a case study of expanded treatment of LTBI among non-U.S.-born individuals in Massachusetts, covering 10% of the target population annually over 2025-2029.

RESULTS

Between 2022 and 2050, TB incidence rates were projected to decline in all states and the District of Columbia. Incidence projections for the year 2050 ranged from 0.03 to 3.8 cases (median 0.95) per 100,000 persons. By 2050, we project that majority (> 50%) of TB will be diagnosed among non-U.S.-born persons in 46 states and the District of Columbia; per state percentages range from 17.4% to 96.7% (median 83.0%). In Massachusetts, expanded testing and treatment for LTBI in this population was projected to reduce cumulative TB cases between 2025 and 2050 by 6.3% and TB-related deaths by 8.4%, relative to base case projections. This intervention had an incremental cost-effectiveness ratio of $180,951 (2020 USD) per quality-adjusted life year gained from the societal perspective.

CONCLUSIONS

Tabby2 allows users to estimate the costs, impact, and cost-effectiveness of different TB prevention approaches for multiple geographic areas in the United States. Expanded testing and treatment for LTBI could accelerate declines in TB incidence in the United States, as demonstrated in the Massachusetts case study.

An approach for improving the quality of country-level TB modelling

Mathematical modelling is increasingly used to inform budgeting and strategic decision-making by national TB programmes. Despite the importance of these decisions, there is currently no mechanism to review and confirm the appropriateness of modelling analyses. We have developed a benchmarking, reporting, and review (BRR) approach and accompanying tools to allow constructive review of country-level TB modelling applications. This approach has been piloted in five modelling applications and the results of this study have been used to revise and finalise the approach. The BRR approach consists of 1) quantitative benchmarks against which model assumptions and results can be compared, 2) standardised reporting templates and review criteria, and 3) a multi-stage review process providing feedback to modellers during the application, as well as a summary evaluation after completion. During the pilot, use of the tools prompted important changes in the approaches taken to modelling. The pilot also identified issues beyond the scope of a review mechanism, such as a lack of empirical evidence and capacity constraints. This approach provides independent evaluation of the appropriateness of modelling decisions during the course of an application, allowing meaningful changes to be made before results are used to inform decision-making. The use of these tools can improve the quality and transparency of country-level TB modelling applications.

Building resource constraints and feasibility considerations in mathematical models for infectious disease: A systematic literature review

•

Mathematical model capabilities to explore complex systems now enable priority-setting to consider local resource constraints.

•

Common objectives of model-based analyses incorporating constraints are to assess real-world feasibility or allocate resources efficiently.

•

Constraints may be incorporated via (i) model-based estimation; (ii) linkage of mathematical and health system models; or (iii) optimisation.

•

Models can then project constrained intervention effects and costs and resource requirement s for delivering interventions at full scale.

•

'Health system constraints' should be systematically defined for routine operationalisation in model-based priority-setting.

Priority setting for infectious disease control is increasingly concerned with physical input constraints and other real-world restrictions on implementation and on the decision process. These health system constraints determine the ‘feasibility’ of interventions and hence impact. However, considering them within mathematical models places additional demands on model structure and relies on data availability. This review aims to provide an overview of published methods for considering constraints in mathematical models of infectious disease.

We systematically searched the literature to identify studies employing dynamic transmission models to assess interventions in any infectious disease and geographical area that included non-financial constraints to implementation. Information was extracted on the types of constraints considered and how these were identified and characterised, as well as on the model structures and techniques for incorporating the constraints.

A total of 36 studies were retained for analysis. While most dynamic transmission models identified were deterministic compartmental models, stochastic models and agent-based simulations were also successfully used for assessing the effects of non-financial constraints on priority setting. Studies aimed to assess reductions in intervention coverage (and programme costs) as a result of constraints preventing successful roll-out and scale-up, and/or to calculate costs and resources needed to relax these constraints and achieve desired coverage levels. We identified three approaches for incorporating constraints within the analyses: (i) estimation within the disease transmission model; (ii) linking disease transmission and health system models; (iii) optimising under constraints (other than the budget).

The review highlighted the viability of expanding model-based priority setting to consider health system constraints. We show strengths and limitations in current approaches to identify and quantify locally-relevant constraints, ranging from simple assumptions to structured elicitation and operational models. Overall, there is a clear need for transparency in the way feasibility is defined as a decision criteria for its systematic operationalisation within models.

Estimating Cost Functions for Resource Allocation Using Transmission Models: A Case Study of Tuberculosis Case Finding in South Africa

OBJECTIVE

Cost functions linked to transmission dynamic models are commonly used to estimate the resources required for infectious disease policies. We present a conceptual and empirical approach for estimating these functions, allowing for nonconstant marginal costs. We aim to expand on the current approach which commonly assumes linearity of cost over scale.

METHODS

We propose a theoretical framework adapted from the field of transport economics. We specify joint functions of production of services within a disease-specific program. We expand these functions to include qualitative insights of program expansion patterns. We present the difference in incremental total costs between an approach assuming constant unit costs and alternative approaches that assume economies of scale, scope and homogeneous or heterogeneous facility recruitment into the programme during scale-up. We illustrate the framework's application in tuberculosis, using secondary data from the literature and routine reporting systems in South Africa.

RESULTS

Economies of capacity and scope substantially change cost estimates over time. Cost data requirements for the proposed approach included standardized and disaggregated unit costs (for a limited number of outputs) and information on the facilities network available to the program.

CONCLUSIONS

The defined functional form will determine the magnitude and shape of costs when outputs and coverage are increasing. This in turn will impact resource allocation decisions. Infectious diseases modelers and economists should use transparent and empirically based cost models for analyses that inform resource allocation decisions. This framework describes a general approach for developing these models.