Incorporating Demand and Supply Constraints into Economic Evaluations in Low-Income and Middle-Income Countries

Adapting Economic Evaluation Methods to Shifting Global Health Priorities: Assessing the Value of Health System Inputs

OBJECTIVES

We highlight the importance of undertaking value assessments for health system inputs if allocative efficiency is to be achieve with health sector resources, with a focus on low- and middle-income countries. However, methodological challenges complicated the application of current economic evaluation techniques to health system input investments.

METHODS

We undertake a review of the literature to examine how assessments of investments in health system inputs have been considered to date, highlighting several studies that have suggested ways to address the methodological issues. Additionally, we surveyed how empirical economic evaluations of health system inputs have approached these issues. Finally, we highlight the steps required to move toward a comprehensive standardized framework for undertaking economic evaluations to make value assessments for investments in health systems.

RESULTS

Although the methodological challenges have been illustrated, a comprehensive framework for value assessments of health system inputs, guiding the evidence required, does not exist. The applied literature of economic evaluations of health system inputs has largely ignored the issues, likely resulting in inaccurate assessments of cost-effectiveness.

CONCLUSIONS

A majority of health sector budgets are spent on health system inputs, facilitating the provision of healthcare interventions. Although economic evaluation methods are a key component in priority setting for healthcare interventions, such methods are less commonly applied to decision making for investments in health system inputs. Given the growing agenda for investments in health systems, a framework will be increasingly required to guide governments and development partners in prioritizing investments in scarce health sector budgets.

How Least Developed to Lower-Middle Income Countries Use Health Technology Assessment: A Scoping Review

Health Technology Assessment (HTA) is a multidisciplinary tool to inform healthcare decision-making. HTA has been implemented in high-income countries (HIC) for several decades but has only recently seen a growing investment in low- and middle-income countries. A scoping review was undertaken to define and compare the role of HTA in least developed and lower middle-income countries (LLMIC). MEDLINE and EMBASE databases were searched from January 2015 to August 2021. A matrix comprising categories on HTA objectives, methods, geographies, and partnerships was used for data extraction and synthesis to present our findings. The review identified 50 relevant articles. The matrix was populated and sub-divided into further categories as appropriate. We highlight topical aspects of HTA, including initiatives to overcome well-documented challenges around data and capacity development, and identify gaps in the research for consideration. Those areas we found to be under-studied or under-utilized included disinvestment, early HTA/implementation, system-level interventions, and cross-sectoral partnerships. We consider broad practical implications for decision-makers and researchers aiming to achieve greater interconnectedness between HTA and health systems and generate recommendations that LLMIC can use for HTA implementation. Whilst HIC may have led the way, LLMIC are increasingly beginning to develop HTA processes to assist in their healthcare decision-making. This review provides a forward-looking model that LLMIC can point to as a reference for their own implementation. We hope this can be seen as timely and useful contributions to optimize the impact of HTA in an era of investment and expansion and to encourage debate and implementation.

Cost-effectiveness of tuberculosis infection prevention and control interventions in South African clinics: a model-based economic evaluation informed by complexity science methods

INTRODUCTION

Nosocomial Mycobacterium tuberculosis (Mtb) transmission substantially impacts health workers, patients and communities. Guidelines for tuberculosis infection prevention and control (TB IPC) exist but implementation in many settings remains suboptimal. Evidence is needed on cost-effective investments to prevent Mtb transmission that are feasible in routine clinic environments.

METHODS

A set of TB IPC interventions was codesigned with local stakeholders using system dynamics modelling techniques that addressed both core activities and enabling actions to support implementation. An economic evaluation of these interventions was conducted at two clinics in KwaZulu-Natal, employing agent-based models of Mtb transmission within the clinics and in their catchment populations. Intervention costs included the costs of the enablers (eg, strengthened supervision, community sensitisation) identified by stakeholders to ensure uptake and adherence.

RESULTS

All intervention scenarios modelled, inclusive of the relevant enablers, cost less than US$200 per disability-adjusted life-year (DALY) averted and were very cost-effective in comparison to South Africa's opportunity cost-based threshold (US$3200 per DALY averted). Two interventions, building modifications to improve ventilation and maximising use of the existing Central Chronic Medicines Dispensing and Distribution system to reduce the number of clinic attendees, were found to be cost saving over the 10-year model time horizon. Incremental cost-effectiveness ratios were sensitive to assumptions on baseline clinic ventilation rates, the prevalence of infectious TB in clinic attendees and future HIV incidence but remained highly cost-effective under all uncertainty analysis scenarios.

CONCLUSION

TB IPC interventions in clinics, including the enabling actions to ensure their feasibility, afford very good value for money and should be prioritised for implementation within the South African health system.

Health system readiness and the implementation of rectal artesunate for severe malaria in sub-Saharan Africa: an analysis of real-world costs and constraints

BACKGROUND

Rectal artesunate, an efficacious pre-referral treatment for severe malaria in children, was deployed at scale in Uganda, Nigeria, and DR Congo. In addition to distributing rectal artesunate, implementation required additional investments in crucial but neglected components in the care for severe malaria. We examined the real-world costs and constraints to rectal artesunate implementation.

METHODS

We collected primary data on baseline health system constraints and subsequent rectal artesunate implementation expenditures. We calculated the equivalent annual cost of rectal artesunate implementation per child younger than 5 years at risk of severe malaria, from a health system perspective, separating neglected routine health system components from incremental costs of rectal artesunate introduction.

FINDINGS

The largest baseline constraints were irregular health worker supervisions, inadequate referral facility worker training, and inadequate malaria commodity supplies. Health worker training and behaviour change campaigns were the largest startup costs, while supervision and supply chain management accounted for most annual routine costs. The equivalent annual costs of preparing the health system for managing severe malaria with rectal artesunate were US$2·63, $2·20, and $4·19 per child at risk and $322, $219, and $464 per child treated in Uganda, Nigeria, and DR Congo, respectively. Strengthening the neglected, routine health system components accounted for the majority of these costs at 71·5%, 65·4%, and 76·4% of per-child costs, respectively. Incremental rectal artesunate costs accounted for the minority remainder.

INTERPRETATION

Although rectal artesunate has been touted as a cost-effective pre-referral treatment for severe malaria in children, its real-world potential is limited by weak and under-financed health system components. Scaling up rectal artesunate or other interventions relying on community health-care providers only makes sense alongside additional, essential health system investments sustained over the long term.

FUNDING

Unitaid.

TRANSLATION

For the French translation of the abstract see Supplementary Materials section.

Using system dynamics modelling to estimate the costs of relaxing health system constraints: a case study of tuberculosis prevention and control interventions in South Africa

Health system constraints are increasingly recognized as an important addition to model-based analyses of disease control interventions, as they affect achievable impact and scale. Enabling activities implemented alongside interventions to relax constraints and reach the intended coverage may incur additional costs, which should be considered in priority setting decisions. We explore the use of group model building, a participatory system dynamics modelling technique, for eliciting information from key stakeholders on the constraints that apply to tuberculosis infection prevention and control processes within primary healthcare clinics in South Africa. This information was used to design feasible interventions, including the necessary enablers to relax existing constraints. Intervention and enabler costs were then calculated at two clinics in KwaZulu-Natal using input prices and quantities from the published literature and local suppliers. Among the proposed interventions, the most inexpensive was retrofitting buildings to improve ventilation (US$1644 per year), followed by maximizing the use of community sites for medication collection among stable patients on antiretroviral therapy (ART; US$3753) and introducing appointments systems to reduce crowding (US$9302). Enablers identified included enhanced staff training, supervision and patient engagement activities to support behaviour change and local ownership. Several of the enablers identified by the stakeholders, such as obtaining building permissions or improving information flow between levels of the health systems, were not amenable to costing. Despite this limitation, an approach to costing rooted in system dynamics modelling can be successfully applied in economic evaluations to more accurately estimate the 'real world' opportunity cost of intervention options. Further empirical research applying this approach to different intervention types (e.g. new preventive technologies or diagnostics) may identify interventions that are not cost-effective in specific contexts based on the size of the required investment in enablers.

Cost of TB services in healthcare facilities in Kenya (No 3)

BACKGROUND: The reduction of Kenya´s TB burden requires improving resource allocation both to and within the National TB, Leprosy and Lung Disease Program (NTLD-P). We aimed to estimate the unit costs of TB services for budgeting by NTLD-P, and allocative efficiency analyses for future National Strategic Plan (NSP) costing.METHODS: We estimated costs of all TB interventions in a sample of 20 public and private health facilities from eight counties. We calculated national-level unit costs from a health provider´s perspective using bottom-up (BU) and top-down (TD) approaches for the financial year 2017-2018 using Microsoft Excel and STATA v16.RESULTS: The mean unit cost for passive case-finding (PCF) was respectively US$38 and US$60 using the BU and TD approaches. The unit BU and TD costs of a 6-month first-line treatment (FLT) course, including monitoring tests, was respectively US$135 and US$160, while those for adult drug-resistant TB (DR-TB) treatment was respectively US$3,230.28 and US$3,926.52 for the 9-month short regimen. Intervention costs highlighted variations between BU and TD approaches. Overall, TD costs were higher than BU, as these are able to capture more costs due to inefficiency (breaks/downtime/leave).CONCLUSION: The activity-based TB unit costs form a comprehensive cost database, and the costing process has built-in capacity within the NTLD-P and international TB research networks, which will inform future TB budgeting processes.

A Rapid Ethnographic Assessment of the Impact of the COVID-19 Pandemic on Mental Health Services Delivery in an Acute Care Medical Emergency Department and Trauma Center

A rapid ethnographic assessment of delivery of mental health services to patients at a Level I trauma center in a major metropolitan hospital undergoing a COVID-19 surge was conducted to assess the challenges involved in services delivery and to compare the experience of delivering services across time. Study participants were patients and providers who interacted with or otherwise were observed by three clinicians engaged in the delivery of care within the Emergency Department (ED) and Trauma Center at Harborview Medical Center from the COVID-19-related "surge" in April to the end of July 2020. Data were collected and analyzed in accordance with the Rapid Assessment Procedures-Informed Clinical Ethnography (RAPICE) protocol. Community and institutional efforts to control the spread of the coronavirus created several challenges to providing mental health services in an acute care setting during the April surge. Most of these challenges were successfully addressed by standardization of infection control protocols, but new challenges emerged including an increase in expenses for infection control and reduction in clinical revenues due to fewer patients, furloughs of mental health services providers and peer specialists in the ED, services not provided or delayed, increased stress due to fear of furloughs or increased workload of those not furloughed, and increases in patients seen with injuries due to risky behavior, violence, and substance use. These findings illustrate the rapidly shifting nature of the pandemic, its impacts on mental health services, and the mitigation efforts of communities and healthcare systems.

The cost and intermediary cost-effectiveness of oral HIV self-test kit distribution across 11 distribution models in South Africa

BACKGROUND

Countries around the world seek innovative ways of closing their remaining gaps towards the target of 95% of people living with HIV (PLHIV) knowing their status by 2030. Offering kits allowing HIV self-testing (HIVST) in private might help close these gaps.

METHODS

We analysed the cost, use and linkage to onward care of 11 HIVST kit distribution models alongside the Self-Testing AfRica Initiative's distribution of 2.2 million HIVST kits in South Africa in 2018/2019. Outcomes were based on telephonic surveys of 4% of recipients; costs on a combination of micro-costing, time-and-motion and expenditure analysis. Costs were calculated from the provider perspective in 2019 US$, as incremental costs in integrated and full costs in standalone models.

RESULTS

HIV positivity among kit recipients was 4%-23%, with most models achieving 5%-6%. Linkage to confirmatory testing and antiretroviral therapy (ART) initiation for those screening positive was 19%-78% and 2%-72% across models. Average costs per HIVST kit distributed varied between $4.87 (sex worker model) and $18.07 (mobile integration model), with differences largely driven by kit volumes. HIVST kit costs (at $2.88 per kit) and personnel costs were the largest cost items throughout. Average costs per outcome increased along the care cascade, with the sex worker network model being the most cost-effective model across metrics used (cost per kit distributed/recipient screening positive/confirmed positive/initiating ART). Cost per person confirmed positive for HIVST was higher than standard HIV testing.

CONCLUSION

HIV self-test distribution models in South Africa varied widely along four characteristics: distribution volume, HIV positivity, linkage to care and cost. Volume was highest in models that targeted public spaces with high footfall (flexible community, fixed point and transport hub distribution), followed by workplace models. Transport hub, workplace and sex worker models distributed kits in the least costly way. Distribution via index cases at facility as well as sex worker network distribution identified the highest number of PLHIV at lowest cost.

How do participatory methods shape policy? Applying a realist approach to the formulation of a new tuberculosis policy in Georgia

OBJECTIVES

This paper presents the iterative process of participatory multistakeholder engagement that informed the development of a new national tuberculosis (TB) policy in Georgia, and the lessons learnt.

METHODS

Guided by realist evaluation methods, a multistakeholder dialogue was organised to elicit stakeholders' assumptions on challenges and possible solutions for better TB control. Two participatory workshops were conducted with key actors, interspersed by reflection meetings within the research team and discussions with policymakers. Using concept mapping and causal mapping techniques, and drawing causal loop diagrams, we visualised how actors understood TB service provision challenges and the potential means by which a results-based financing (RBF) policy could address these.

SETTING

The study was conducted in Tbilisi, Georgia.

PARTICIPANTS

A total of 64 key actors from the Ministry of Labour, Health and Social Affairs, staff of the Global Fund to Fight AIDS, TB and Malaria Georgia Project, the National Centre for Disease Control and Public Health, the National TB programme, TB service providers and members of the research team were involved in the workshops.

RESULTS

Findings showed that beyond provider incentives, additional policy components were necessary. These included broadening the incentive package to include institutional and organisational incentives, retraining service providers, clear redistribution of roles to support an integrated care model, and refinement of monitoring tools. Health system elements, such as effective referral systems and health information systems were highlighted as necessary for service improvement.

CONCLUSIONS

Developing policies that address complex issues requires methods that facilitate linkages between multiple stakeholders and between theory and practice. Such participatory approaches can be informed by realist evaluation principles and visually facilitated by causal loop diagrams. This approach allowed us leverage stakeholders' knowledge and expertise on TB service delivery and RBF to codesign a new policy.

Program Evaluation of Population- and System-Level Policies: Evidence for Decision Making

BACKGROUND

Policy evaluations often focus on ex post estimation of causal effects on short-term surrogate outcomes. The value of such information is limited for decision making, as the failure to reflect policy-relevant outcomes and disregard for opportunity costs prohibits the assessment of value for money. Further, these evaluations do not always consider all relevant evidence, other courses of action, or decision uncertainty.

METHODS

In this article, we explore how policy evaluation could better meet the needs of decision making. We begin by defining the evidence required to inform decision making. We then conduct a literature review of challenges in evaluating policies. Finally, we highlight potential methods available to help address these challenges.

RESULTS

The evidence required to inform decision making includes the impacts on the policy-relevant outcomes, the costs and associated opportunity costs, and the consequences of uncertainty. Challenges in evaluating health policies are described using 8 categories: 1) valuation space; 2) comparators; 3) time of evaluation; 4) mechanisms of action; 5) effects; 6) resources, constraints, and opportunity costs; 7) fidelity, adaptation, and level of implementation; and 8) generalizability and external validity. Methods from a broad set of disciplines are available to improve policy evaluation, relating to causal inference, decision-analytic modeling, theory of change, realist evaluation, and structured expert elicitation.

LIMITATIONS

The targeted review may not identify all possible challenges, and the methods covered are not exhaustive.

CONCLUSIONS

Evaluations should provide appropriate evidence to inform decision making. There are challenges in evaluating policies, but methods from multiple disciplines are available to address these challenges.

IMPLICATIONS

Evaluators need to carefully consider the decision being informed, the necessary evidence to inform it, and the appropriate methods.[Box: see text].

Building resource constraints and feasibility considerations in mathematical models for infectious disease: A systematic literature review

•

Mathematical model capabilities to explore complex systems now enable priority-setting to consider local resource constraints.

•

Common objectives of model-based analyses incorporating constraints are to assess real-world feasibility or allocate resources efficiently.

•

Constraints may be incorporated via (i) model-based estimation; (ii) linkage of mathematical and health system models; or (iii) optimisation.

•

Models can then project constrained intervention effects and costs and resource requirement s for delivering interventions at full scale.

•

'Health system constraints' should be systematically defined for routine operationalisation in model-based priority-setting.

Priority setting for infectious disease control is increasingly concerned with physical input constraints and other real-world restrictions on implementation and on the decision process. These health system constraints determine the ‘feasibility’ of interventions and hence impact. However, considering them within mathematical models places additional demands on model structure and relies on data availability. This review aims to provide an overview of published methods for considering constraints in mathematical models of infectious disease.

We systematically searched the literature to identify studies employing dynamic transmission models to assess interventions in any infectious disease and geographical area that included non-financial constraints to implementation. Information was extracted on the types of constraints considered and how these were identified and characterised, as well as on the model structures and techniques for incorporating the constraints.

A total of 36 studies were retained for analysis. While most dynamic transmission models identified were deterministic compartmental models, stochastic models and agent-based simulations were also successfully used for assessing the effects of non-financial constraints on priority setting. Studies aimed to assess reductions in intervention coverage (and programme costs) as a result of constraints preventing successful roll-out and scale-up, and/or to calculate costs and resources needed to relax these constraints and achieve desired coverage levels. We identified three approaches for incorporating constraints within the analyses: (i) estimation within the disease transmission model; (ii) linking disease transmission and health system models; (iii) optimising under constraints (other than the budget).

The review highlighted the viability of expanding model-based priority setting to consider health system constraints. We show strengths and limitations in current approaches to identify and quantify locally-relevant constraints, ranging from simple assumptions to structured elicitation and operational models. Overall, there is a clear need for transparency in the way feasibility is defined as a decision criteria for its systematic operationalisation within models.

Costs and outcomes of active and passive case detection for visceral leishmaniasis (Kala-Azar) to inform elimination strategies in Bihar, India

BACKGROUND

Effective case identification strategies are fundamental to capturing the remaining visceral leishmaniasis (VL) cases in India. To inform government strategies to reach and sustain elimination benchmarks, this study presents costs of active- and passive- case detection (ACD and PCD) strategies used in India's most VL-endemic state, Bihar, with a focus on programme outcomes stratified by district-level incidence.

METHODS

Expenditure analysis was complemented by onsite micro-costing to compare the cost of PCD in hospitals alongside index case-based ACD and a combination of blanket (house-to-house) and camp ACD from January to December 2018. From the provider's perspective, a cost analysis evaluated the overall programme cost of each activity, the cost per case detected, and the cost of scaling up ACD.

RESULTS

During 2018, index case-based ACD, blanket and camp ACD, and PCD reported 1,497, 131, and 1,983 VL-positive cases at a unit cost of $522.81, $4,186.81, and $246.79, respectively. In high endemic districts, more VL cases were identified through PCD while in meso- and low-endemic districts more cases were identified through ACD. The cost of scaling up ACD to identify 3,000 additional cases ranged from $1.6-4 million, depending on the extent to which blanket and camp ACD was relied upon.

CONCLUSION

Cost per VL test conducted (rather than VL-positive case identified) may be a better metric estimating unit costs to scale up ACD in Bihar. As more VL cases were identified in meso-and low-endemic districts through ACD than PCD, health authorities in India should consider bolstering ACD in these areas. Blanket and camp ACD identified fewer cases at a higher unit cost than index case-based ACD. However, the value of detecting additional VL cases early outweighs long-term costs for reaching and sustaining VL elimination benchmarks in India.

Informing Balanced Investment in Services and Health Systems: A Case Study of Priority Setting for Tuberculosis Interventions in South Africa

OBJECTIVES

Health systems face nonfinancial constraints that can influence the opportunity cost of interventions. Empirical methods to explore their impact, however, are underdeveloped. We develop a conceptual framework for defining health system constraints and empirical estimation methods that rely on routine data. We then present an empirical approach for incorporating nonfinancial constraints in cost-effectiveness models of health benefit packages for the health sector.

METHODS

We illustrate the application of this approach through a case study of defining a package of services for tuberculosis case-finding in South Africa. An economic model combining transmission model outputs with unit costs was developed to examine the cost-effectiveness of alternative screening and diagnostic algorithms. Constraints were operationalized as restrictions on achievable coverage based on: (1) financial resources; (2) human resources; and (3) policy constraints around diagnostics purchasing. Cost-effectiveness of the interventions was assessed under one "unconstrained" and several "constrained" scenarios. For the unconstrained scenario, incremental cost-effectiveness ratios were estimated with and without the costs of "relaxing" constraints.

RESULTS

We find substantial differences in incremental cost-effectiveness ratios across scenarios, leading to variations in the decision rules for prioritizing interventions. In constrained scenarios, the limiting factor for most interventions was not financial, but rather the availability of human resources.

CONCLUSIONS

We find that optimal prioritization among different tuberculosis control strategies in South Africa is influenced by whether and how constraints are taken into consideration. We thus demonstrate both the importance and feasibility of considering nonfinancial constraints in health sector resource allocation models.

SEEP-CI: A Structured Economic Evaluation Process for Complex Health System Interventions

The economic evaluation of health system interventions is challenging, and methods guidance on how to respond to these challenges is lacking. The REACHOUT consortium developed and evaluated complex interventions for community health program quality improvement in six countries in Africa and Asia. Reflecting on the challenges we faced in conducting an economic evaluation alongside REACHOUT, we developed a Structured Economic Evaluation Process for Complex Health System Interventions (SEEP-CI). The SEEP-CI aims to establish the threshold effect size that would justify investment in a complex intervention, and provide an assessment to a decision-maker of how likely it is that the intervention can achieve this impact. We illustrate how the SEEP-CI could have been applied to REACHOUT to identify outcomes where the intervention might have impact and causal mechanisms, through which that impact might occur, guide data collection by focusing on proximal outcomes most likely to illustrate the effectiveness of the intervention, identify the size of health gain required to justify investment in the intervention, and indicate the assumptions required to accept that such health gains are credible. Further research is required to determine the feasibility and acceptability of the SEEP-CI, and the contexts in which it could be used.

A comprehensive framework for considering additional unintended consequences in economic evaluation

Background

In recent years there has been a growth in economic evaluations that consider indirect health benefits to populations due to advances in mathematical modeling. In addition, economic evaluations guidelines have suggested the inclusion of impact inventories to include non-health direct and indirect consequences. We aim to bring together this literature, together with the broader literature on internalities and externalities to propose a comprehensive approach for analysts to identify and characterize all unintended consequences in economic evaluations.

Methods

We present a framework to assist analysts identify and characterize additional costs and effects beyond that of direct health impact primarily intended to be influenced by the intervention/technology. We build on previous checklists to provide analysts with a comprehensive framework to justify the inclusion or exclusion of effects, supporting the use of current guidelines, to ensure any unintended effects are considered. We illustrate this framework with examples from immunization. These were identified from a previous systematic review, PhD thesis work, and general search scoping in PubMed databases.

Results

We present a comprehensive framework to consider additional consequences, exemplified by types and categories. We bring this and other guidance together to assist analysts identify possible unintended consequences whether taking a provider or societal perspective.

Conclusions

Although there are many challenges ahead to standardize the inclusion of additional consequences in economic evaluation, we hope by moving beyond generic statements to reporting against a comprehensive framework of additional effects we can support further consistency in this aspect of cost-effectiveness analysis going forward.

A Systematic Review of Methodological Variation in Healthcare Provider Perspective Tuberculosis Costing Papers Conducted in Low- and Middle-Income Settings, Using An Intervention-Standardised Unit Cost Typology

BACKGROUND

There is a need for easily accessible tuberculosis unit cost data, as well as an understanding of the variability of methods used and reporting standards of that data.

OBJECTIVE

The aim of this systematic review was to descriptively review papers reporting tuberculosis unit costs from a healthcare provider perspective looking at methodological variation; to assess quality using a study quality rating system and machine learning to investigate the indicators of reporting quality; and to identify the data gaps to inform standardised tuberculosis unit cost collection and consistent principles for reporting going forward.

METHODS

We searched grey and published literature in five sources and eight databases, respectively, using search terms linked to cost, tuberculosis and tuberculosis health services including tuberculosis treatment and prevention. For inclusion, the papers needed to contain empirical unit cost estimates for tuberculosis interventions from low- and middle-income countries, with reference years between 1990 and 2018. A total of 21,691 papers were found and screened in a phased manner. Data were extracted from the eligible papers into a detailed Microsoft Excel tool, extensively cleaned and analysed with R software (R Project, Vienna, Austria) using the user interface of RStudio. A study quality rating was applied to the reviewed papers based on the inclusion or omission of a selection of variables and their relative importance. Following this, machine learning using a recursive partitioning method was utilised to construct a classification tree to assess the reporting quality.

RESULTS

This systematic review included 103 provider perspective papers with 627 unit costs (costs not presented here) for tuberculosis interventions among a total of 140 variables. The interventions covered were active, passive and intensified case finding; tuberculosis treatment; above-service costs; and tuberculosis prevention. Passive case finding is the detection of tuberculosis cases where individuals self-identify at health facilities; active case finding is detection of cases of those not in health facilities, such as through outreach; and intensified case finding is detection of cases in high-risk populations. There was heterogeneity in some of the reported methods used such cost allocation, amortisation and the use of top-down, bottom-up or mixed approaches to the costing. Uncertainty checking through sensitivity analysis was only reported on by half of the papers (54%), while purposive and convenience sampling was reported by 72% of papers. Machine learning indicated that reporting on 'Intervention' (in particular), 'Urbanicity' and 'Site Sampling', were the most likely indicators of quality of reporting. The largest data gap identified was for tuberculosis vaccination cost data, the Bacillus Calmette-Guérin (BCG) vaccine in particular. There is a gap in available unit costs for 12 of 30 high tuberculosis burden countries, as well as for the interventions of above-service costs, tuberculosis prevention, and active and intensified case finding.

CONCLUSION

Variability in the methods and reporting used makes comparison difficult and makes it hard for decision makers to know which unit costs they can trust. The study quality rating system used in this review as well as the classification tree enable focus on specific reporting aspects that should improve variability and increase confidence in unit costs. Researchers should endeavour to be explicit and transparent in how they cost interventions following the principles as laid out in the Global Health Cost Consortium's Reference Case for Estimating the Costs of Global Health Services and Interventions, which in turn will lead to repeatability, comparability and enhanced learning from others.

Integrating Economic Evaluation and Implementation Science to Advance the Global HIV Response

BACKGROUND

Numerous cost-effectiveness analyses have indicated good value for money from a wide array of interventions for treatment and prevention of HIV/AIDS. There is limited evidence, however, regarding how cost-effectiveness information contributes to better decision-making around investment and action in the global HIV response.

METHODS

We review challenges for economic evaluation relevant to the global HIV response and consider how the practice of cost-effectiveness analysis could integrate approaches and insights from implementation science to enhance the impact and efficiency of HIV investments.

RESULTS

In light of signals that cost-effectiveness analyses may be vulnerable to systematic bias toward overly optimistic conclusions, we emphasize two priorities for advancing the field of economic evaluation in HIV/AIDS and more broadly in global health: (1) systematic reevaluation of the cost-effectiveness literature with reference to ex-post empirical evidence on costs and effects in real-world programs and (2) development and adoption of good-practice guidelines for incorporating implementation and delivery aspects into economic evaluations. Toward the latter aim, we propose an integrative approach that focuses on comparative evaluation of strategies, which specify both technologies/interventions as well as the delivery platforms, complementary interventions, and actions needed to increase coverage, quality, and uptake of those technologies/interventions. Specific recommendations draw on several existing implementation science models that provide systematic frameworks for understanding implementation barriers and enablers, designing and choosing specific implementation and policy actions, and evaluating outcomes.

DISCUSSION

These preliminary steps aimed at bridging the divide between economic evaluation and implementation science can help to advance the practice of economic evaluation toward a science of comparative strategy evaluation.

Managing Uncertainties Due to Limited Evidence in Economic Evaluations of Novel Anti-Tuberculosis Regimens: A Systematic Review

BACKGROUND

Limited evidence for the implementation of new health technologies in low- and middle-income countries (LMICs) may lead to uncertainties in economic evaluations and cause the evaluations to produce inaccurate information for decision making. We performed a systematic review of economic evaluations on implementing new short-course regimens (SCR) for drug-sensitive and drug-resistant tuberculosis (TB), to explore how uncertainties due to the limited evidence in the studies were dealt with and to identify useful information for decision making from these studies.

METHODS

We searched in electronic databases PubMed, EMBASE, NHSEED, and CEA registry for economic evaluations addressing the implementation of new anti-TB SCRs in LMICs published until September 2018. We included studies addressing both the cost and outcomes of implementing a new regimen for drug-sensitive and drug-resistant TB with a shorter treatment duration than the currently used regimens. The quality of the included studies was assessed using The Consensus Health Economic Criteria checklist. We extracted information from the included studies on uncertainties and how they were managed. The management of uncertainties was compared with approaches used in early health technology assessments (HTAs), including sensitivity analyses and pragmatic scenario analyses. We extracted information that could be useful for decision making such as cost-effectiveness conclusions, and barriers to implementing the intervention.

RESULTS

Four of the 322 studies found in the search met the eligibility criteria. Three studies were model-based studies that investigated the cost effectiveness of a new first-line SCR. One study was an empirical study investigating the cost effectiveness of new regimens for drug-resistant TB. The model-based studies addressed uncertainties due to limited evidence through various sensitivity analyses as in early HTAs. They performed a deterministic sensitivity analysis and found the main drivers of the cost-effectiveness outcomes, that is, the rate of treatment default and treatment delivery costs. Additionally, two of the model-based studies performed a pragmatic scenario analysis and found a potential barrier to implementing the new first-line SCR, that is, a weak health system with a low TB care utilization rate. The empirical study only performed a few scenario analyses with different regimen prices and volumes of TB care utilization. Therefore, the study could only provide information on the main cost drivers.

CONCLUSION

Using an approach similar to that used in early HTAs, where uncertainties due to the limited evidence are rigorously explored upfront, the economic evaluations could inform not only the decision to implement the intervention but also how to manage risks and implementation barriers.

The cost of providing and scaling up surgery: a comparison of a district hospital and a referral hospital in Zambia

The lack of access to quality-assured surgery in rural parts of sub-Saharan Africa, where the numbers of trained health workers are often insufficient, presents challenges for national governments. The case for investing in scaling up surgical systems in low-resource settings is 3-fold: the potential beneficial impact on a large proportion of the global burden of disease; better access for rural populations who have the greatest unmet need; and the economic case. The economic losses from untreated surgical conditions far exceed any expenditure that would be required to scale up surgical care. We identified the resources used in delivering surgery at a rural district-level hospital and an urban based referral hospital in Zambia and calculated their cost through a combination of bottom-up costing and step-down accounting. Surgery performed at the referral hospital is ∼50% more expensive compared with the district hospital, mostly because of the higher cost of hospital stay. The low bed occupancy rates at the two hospitals suggest underutilization of the capacity, and/or missing elements of needed capacity, to conduct surgery. Nevertheless, our study confirms that scaling up district-level surgery makes sense, through bringing economies of scale, while acknowledging the need for more comprehensive assessments and costing of capacity constraints. We quantified the economies of scale under different scaling scenarios. If surgery at the district hospital was scaled up by 10, 20 or 50%, the total cost of surgery would increase proportionately less than that, i.e. by 6, 12 and 30%, respectively. If this were to lead to less demand for surgery at the referral hospital, say 10% less surgery, it would result in a reduction of 2.7% in the total cost. Although the health system as a whole would benefit, the referring hospitals would not derive the full economic benefit, unless Government increased resources for district-level surgery.

Rapid, point-of-care diagnosis of tuberculosis with novel Truenat assay: Cost-effectiveness analysis for India's public sector

Background

Truenat is a novel molecular assay that rapidly detects tuberculosis (TB) and rifampicin-resistance. Due to the portability of its battery-powered testing platform, it may be valuable in peripheral healthcare settings in India.

Methods

Using a microsimulation model, we compared four TB diagnostic strategies for HIV-negative adults with presumptive TB: (1) sputum smear microscopy in designated microscopy centers (DMCs) (SSM); (2) Xpert MTB/RIF in DMCs (Xpert); (3) Truenat in DMCs (Truenat DMC); and (4) Truenat for point-of-care testing in primary healthcare facilities (Truenat POC). We projected life expectancy, costs, incremental cost-effectiveness ratios (ICERs), and 5-year budget impact of deploying Truenat POC in India’s public sector. We defined a strategy “cost-effective” if its ICER was <US$990/year-of-life saved (YLS). Model inputs included: TB prevalence, 15% (among those not previously treated for TB) and 27% (among those previously treated for TB); sensitivity for TB detection, 89% (Xpert) and 86% (Truenat); per test cost, $12.63 (Xpert) and $13.20 (Truenat); and linkage-to-care after diagnosis, 84% (DMC) and 95% (POC). We varied these parameters in sensitivity analyses.

Results

Compared to SSM, Truenat POC increased life expectancy by 0.39 years and was cost-effective (ICER $210/YLS). Compared to Xpert, Truenat POC increased life expectancy by 0.08 years due to improved linkage-to-care and was cost-effective (ICER $120/YLS). In sensitivity analysis, the cost-effectiveness of Truenat POC, relative to Xpert, depended on the diagnostic sensitivity of Truenat and linkage-to-care with Truenat. Deploying Truenat POC instead of Xpert increased 5-year expenditures by $270 million, due mostly to treatment costs. Limitations of our study include uncertainty in Truenat’s sensitivity for TB and not accounting for the “start-up” costs of implementing Truenat in the field.

Conclusions

Used at the point-of-care in India, Truenat for TB diagnosis should improve linkage-to-care, increase life expectancy, and be cost-effective compared with smear microscopy or Xpert.

The influence of constraints on the efficient allocation of resources for HIV prevention

OBJECTIVE

To investigate how 'real-world' constraints on the allocative and technical efficiency of HIV prevention programmes affect resource allocation and the number of infections averted.

DESIGN

Epidemiological modelling and economic analyses in Benin, South Africa and Tanzania.

METHODS

We simulated different HIV prevention programmes, and first determined the most efficient allocation of resources, in which the HIV prevention budget is shared among specific interventions, risk-groups and provinces to maximize the number of infections averted. We then identified the efficient allocation of resources and achievable impact given the following constraints to allocative efficiency: earmarking [provinces with budgets fund pre-exposure prophylaxis (PrEP) for low-risk women first], meeting targets [provinces with budgets fund universal test-and-treat (UTT) first] and minimizing changes in the geographical distribution of funds. We modelled technical inefficiencies as a reduction in the coverage of PrEP or UTT, which were factored into the resource allocation process or took effect following the allocation. Each scenario was investigated over a range of budgets, such that the impact reaches its maximum.

RESULTS

The 'earmarking', 'meeting targets' and 'minimizing change' constraints reduce the potential impact of HIV prevention programmes, but at the higher budgets these constraints have little to no effect (approximately 35 billion US$ in Tanzania). Over-estimating technical efficiency can result in a loss of impact compared to what would be possible if technical efficiencies were known accurately.

CONCLUSION

Failing to account for constraints on allocative and technical efficiency can result in the overestimation of the health gains possible, and for technical inefficiencies the allocation of an inefficient strategy.

Cost Effectiveness of Mobile Health for Antenatal Care and Facility Births in Nigeria

Background:

The use of mobile technology in the health sector, often referred to as mHealth, is an innovation that is being used in countries to improve health outcomes and increase and improve both the demand and supply of health care services. This study assesses the actual cost-effectiveness of initiating and implementing the use of the mHealth as a supply side job aid for antenatal care. The study also estimates the cost-effectiveness ratio if mHealth was also used to encourage and track women through facility delivery.

Methods:

The methodology utilized a retrospective, micro-costing technique to extract costing data from health facilities and administrative offices to estimate the costs of implementing the mHealth antenatal care program and estimate the cost of facility delivery for those that used the antenatal care services in the year 2014. Five different costing tools were developed to assist in the costing analysis.

Findings:

The results show that the provision of tetanus toxoid vaccination and malaria prophylaxis during pregnancy and improved labor and delivery during facility delivery contributed the most to mortality reductions for women, neonates and stillbirths in mHealth facilities versus non-mHealth facilities. The cost-effectiveness ratio of this program for antenatal care and no demand-side generation for facility delivery is US$13,739 per life saved. The cost-effectiveness ratio adding in an additional demand-side generation for facility births reduces to US$9,806 per life saved.

Conclusion:

These results show that mHealth programs are inexpensive and save a number of lives for the dollar investment and could save additional lives and funds if women were also encouraged to seek facility delivery.

Cost Effectiveness of Mobile Health for Antenatal Care and Facility Births in Nigeria

BACKGROUND

The use of mobile technology in the health sector, often referred to as mHealth, is an innovation that is being used in countries to improve health outcomes and increase and improve both the demand and supply of health care services. This study assesses the actual cost-effectiveness of initiating and implementing the use of the mHealth as a supply side job aid for antenatal care. The study also estimates the cost-effectiveness ratio if mHealth was also used to encourage and track women through facility delivery.

METHODS

The methodology utilized a retrospective, micro-costing technique to extract costing data from health facilities and administrative offices to estimate the costs of implementing the mHealth antenatal care program and estimate the cost of facility delivery for those that used the antenatal care services in the year 2014. Five different costing tools were developed to assist in the costing analysis.

FINDINGS

The results show that the provision of tetanus toxoid vaccination and malaria prophylaxis during pregnancy and improved labor and delivery during facility delivery contributed the most to mortality reductions for women, neonates and stillbirths in mHealth facilities versus non-mHealth facilities. The cost-effectiveness ratio of this program for antenatal care and no demand-side generation for facility delivery is US$13,739 per life saved. The cost-effectiveness ratio adding in an additional demand-side generation for facility births reduces to US$9,806 per life saved.

CONCLUSION

These results show that mHealth programs are inexpensive and save a number of lives for the dollar investment and could save additional lives and funds if women were also encouraged to seek facility delivery.

Empirical estimation of resource constraints for use in model-based economic evaluation: an example of TB services in South Africa

BACKGROUND

Evidence on the relative costs and effects of interventions that do not consider 'real-world' constraints on implementation may be misleading. However, in many low- and middle-income countries, time and data scarcity mean that incorporating health system constraints in priority setting can be challenging.

METHODS

We developed a 'proof of concept' method to empirically estimate health system constraints for inclusion in model-based economic evaluations, using intensified case-finding strategies (ICF) for tuberculosis (TB) in South Africa as an example. As part of a strategic planning process, we quantified the resources (fiscal and human) needed to scale up different ICF strategies (cough triage and WHO symptom screening). We identified and characterised three constraints through discussions with local stakeholders: (1) financial constraint: potential maximum increase in public TB financing available for new TB interventions; (2) human resource constraint: maximum current and future capacity among public sector nurses that could be dedicated to TB services; and (3) diagnostic supplies constraint: maximum ratio of Xpert MTB/RIF tests to TB notifications. We assessed the impact of these constraints on the costs of different ICF strategies.

RESULTS

It would not be possible to reach the target coverage of ICF (as defined by policy makers) without addressing financial, human resource and diagnostic supplies constraints. The costs of addressing human resource constraints is substantial, increasing total TB programme costs during the period 2016-2035 by between 7% and 37% compared to assuming the expansion of ICF is unconstrained, depending on the ICF strategy chosen.

CONCLUSIONS

Failure to include the costs of relaxing constraints may provide misleading estimates of costs, and therefore cost-effectiveness. In turn, these could impact the local relevance and credibility of analyses, thereby increasing the risk of sub-optimal investments.

Cost-Effectiveness Thresholds: the Past, the Present and the Future

Cost-effectiveness (CE) thresholds are being discussed more frequently and there have been many new developments in this area; however, there is a lack of understanding about what thresholds mean and their implications. This paper provides an overview of the CE threshold literature. First, the meaning of a CE threshold and the key assumptions involved (perfect divisibility, marginal increments in budget, etc.) are highlighted using a hypothetical example, and the use of historic/heuristic estimates of the threshold is noted along with their limitations. Recent endeavours to estimate the empirical value of the thresholds, both from the supply side and the demand side, are then presented. The impact on CE thresholds of future directions for the field, such as thresholds across sectors and the incorporation of multiple criteria beyond quality-adjusted life-years as a measure of 'value', are highlighted. Finally, a number of common issues and misconceptions associated with CE thresholds are addressed.

Health care input constraints and cost effectiveness analysis decision rules

Results of cost effectiveness analyses (CEA) studies are most useful for decision makers if they face only one constraint: the health care budget. However, in practice, decision makers wishing to use the results of CEA studies may face multiple resource constraints relating to, for instance, constraints in health care inputs such as a shortage of skilled labour. The presence of multiple resource constraints influences the decision rules of CEA and limits the usefulness of traditional CEA studies for decision makers. The goal of this paper is to illustrate how results of CEA can be interpreted and used in case a decision maker faces a health care input constraint. We set up a theoretical model describing the optimal allocation of the health care budget in the presence of a health care input constraint. Insights derived from that model were used to analyse a stylized example based on a decision about a surgical robot as well as a published cost effectiveness study on eye care services in Zambia. Our theoretical model shows that applying default decision rules in the presence of a health care input constraint leads to suboptimal decisions but that there are ways of preserving the traditional decision rules of CEA by reweighing different cost categories. The examples illustrate how such adjustments can be made, and makes clear that optimal decisions depend crucially on such adjustments. We conclude that it is possible to use the results of cost effectiveness studies in the presence of health care input constraints if results are properly adjusted.

Large-scale delivery of seasonal malaria chemoprevention to children under 10 in Senegal: an economic analysis

Seasonal Malaria Chemoprevention (SMC) is recommended for children under 5 in the Sahel and sub-Sahel. The burden in older children may justify extending the age range, as has been done effectively in Senegal. We examine costs of door-to-door SMC delivery to children up to 10 years by community health workers (CHWs). We analysed incremental financial and economic costs at district level and below from a health service perspective. We examined project accounts and prospectively collected data from 405 CHWs, 46 health posts, and 4 district headquarters by introducing questionnaires in advance and completing them after each monthly implementation round. Affordability was explored by comparing financial costs of SMC to relevant existing health expenditure levels. Costs were disaggregated by administration month and by health service level. We used linear regression models to identify factors associated with cost variation between health posts. The financial cost to administer SMC to 180 000 children over one malaria season, reaching ∼93% of children with all three intended courses of SMC was $234 549 (constant 2010 USD) or $0.50 per monthly course administered. Excluding research-participation incentives, the financial cost was $0.32 per resident (all ages) in the catchment area, which is 1.2% of Senegal's general government expenditure on health per capita. Economic costs were 18.7% higher than financial costs at $278 922 or $0.59 per course administered and varied widely between health posts, from $0.38 to $2.74 per course administered. Substantial economies of scale across health posts were found, with the smallest health posts incurring highest average costs per monthly course administered. SMC for children up to 10 is likely to be affordable, particularly where it averts substantial curative care costs. Estimates of likely costs and cost-effectiveness of SMC in other contexts must account for variation in average costs across delivery months and health posts.

Cost-effectiveness of Xpert MTB/RIF for tuberculosis diagnosis in South Africa: a real-world cost analysis and economic evaluation

BACKGROUND

In 2010 a new diagnostic test for tuberculosis, Xpert MTB/RIF, received a conditional programmatic recommendation from WHO. Several model-based economic evaluations predicted that Xpert would be cost-effective across sub-Saharan Africa. We investigated the cost-effectiveness of Xpert in the real world during national roll-out in South Africa.

METHODS

For this real-world cost analysis and economic evaluation, we applied extensive primary cost and patient event data from the XTEND study, a pragmatic trial examining Xpert introduction for people investigated for tuberculosis in 40 primary health facilities (20 clusters) in South Africa enrolled between June 8, and Nov 16, 2012, to estimate the costs and cost per disability-adjusted life-year averted of introducing Xpert as the initial diagnostic test for tuberculosis, compared with sputum smear microscopy (the standard of care).

FINDINGS

The mean total cost per study participant for tuberculosis investigation and treatment was US$312·58 (95% CI 252·46-372·70) in the Xpert group and $298·58 (246·35-350·82) in the microscopy group. The mean health service (provider) cost per study participant was $168·79 (149·16-188·42) for the Xpert group and $160·46 (143·24-177·68) for the microscopy group of the study. Considering uncertainty in both cost and effect using a wide range of willingness to pay thresholds, we found less than 3% probability that Xpert introduction improved the cost-effectiveness of tuberculosis diagnostics.

INTERPRETATION

After analysing extensive primary data collection during roll-out, we found that Xpert introduction in South Africa was cost-neutral, but found no evidence that Xpert improved the cost-effectiveness of tuberculosis diagnosis. Our study highlights the importance of considering implementation constraints, when predicting and evaluating the cost-effectiveness of new tuberculosis diagnostics in South Africa.

FUNDING

Bill & Melinda Gates Foundation.

Policies for Use of Real-World Data in Health Technology Assessment (HTA): A Comparative Study of Six HTA Agencies

BACKGROUND

Randomized controlled trials provide robust data on the efficacy of interventions rather than on effectiveness. Health technology assessment (HTA) agencies worldwide are thus exploring whether real-world data (RWD) may provide alternative sources of data on effectiveness of interventions. Presently, an overview of HTA agencies' policies for RWD use in relative effectiveness assessments (REA) is lacking.

OBJECTIVES

To review policies of six European HTA agencies on RWD use in REA of drugs. A literature review and stakeholder interviews were conducted to collect information on RWD policies for six agencies: the Dental and Pharmaceutical Benefits Agency (Sweden), the National Institute for Health and Care Excellence (United Kingdom), the Institute for Quality and Efficiency in Healthcare (Germany), the High Authority for Health (France), the Italian Medicines Agency (Italy), and the National Healthcare Institute (The Netherlands). The following contexts for RWD use in REA of drugs were reviewed: initial reimbursement discussions, pharmacoeconomic analyses, and conditional reimbursement schemes. We identified 13 policy documents and 9 academic publications, and conducted 6 interviews.

RESULTS

Policies for RWD use in REA of drugs notably differed across contexts. Moreover, policies differed between HTA agencies. Such variations might discourage the use of RWD for HTA.

CONCLUSIONS

To facilitate the use of RWD for HTA across Europe, more alignment of policies seems necessary. Recent articles and project proposals of the European network of HTA may provide a starting point to achieve this.

Methodological Issues to Consider When Collecting Data to Estimate Poverty Impact in Economic Evaluations in Low-income and Middle-income Countries

Out-of-pocket spending is increasingly recognized as an important barrier to accessing health care, particularly in low-income and middle-income countries (LMICs) where a large portion of health expenditure comes from out-of-pocket payments. Emerging universal healthcare policies prioritize reduction of poverty impact such as catastrophic and impoverishing healthcare expenditure. Poverty impact is therefore increasingly evaluated alongside and within economic evaluations to estimate the impact of specific health interventions on poverty. However, data collection for these metrics can be challenging in intervention-based contexts in LMICs because of study design and practical limitations. Using a set of case studies, this letter identifies methodological challenges in collecting patient cost data in LMIC contexts. These components are presented in a framework to encourage researchers to consider the implications of differing approaches in data collection and to report their approach in a standardized and transparent way.

Comparison of Economic Evaluation Methods Across Low-income, Middle-income and High-income Countries: What are the Differences and Why?

There are marked differences in methods used for undertaking economic evaluations across low-income, middle-income, and high-income countries. We outline the most apparent dissimilarities and reflect on their underlying reasons. We randomly sampled 50 studies from each of three country income groups from a comprehensive database of 2844 economic evaluations published between January 2012 and May 2014. Data were extracted on ten methodological areas: (i) availability of guidelines; (ii) research questions; (iii) perspective; (iv) cost data collection methods; (v) cost data analysis; (vi) outcome measures; (vii) modelling techniques; (viii) cost-effectiveness thresholds; (ix) uncertainty analysis; and (x) applicability. Comparisons were made across income groups and odds ratios calculated. Contextual heterogeneity rightly drives some of the differences identified. Other differences appear less warranted and may be attributed to variation in government health sector capacity, in health economics research capacity and in expectations of funders, journals and peer reviewers. By highlighting these differences, we seek to start a debate about the underlying reasons why they have occurred and to what extent the differences are conducive for methodological advancements. We suggest a number of specific areas in which researchers working in countries of differing environments could learn from one another.

Economic Evaluation in Global Perspective: A Bibliometric Analysis of the Recent Literature

We present a bibliometric analysis of recently published full economic evaluations of health interventions and reflect critically on the implications of our findings for this growing field. We created a database drawing on 14 health, economic, and/or general literature databases for articles published between 1 January 2012 and 3 May 2014 and identified 2844 economic evaluations meeting our criteria. We present findings regarding the sensitivity, specificity, and added value of searches in the different databases. We examine the distribution of publications between countries, regions, and health areas studied and compare the relative volume of research with disease burden. We analyse authors' country and institutional affiliations, journals and journal type, language, and type of economic evaluation conducted. More than 1200 economic evaluations were published annually, of which 4% addressed low-income countries, 4% lower-middle-income countries, 14% upper-middle-income countries, and 83% high-income countries. Across country income levels, 53, 54, 86, and 100% of articles, respectively, included an author based in a country within the income level studied. Biomedical journals published 74% of economic evaluations. The volume of research across health areas correlates more closely with disease burden in high-income than in low-income and middle-income countries. Our findings provide an empirical basis for further study on methods, research prioritization, and capacity development in health economic evaluation.