Cost-Effectiveness Thresholds in Global Health: Taking a Multisectoral Perspective

A new framework for developing and evaluating complex interventions: update of Medical Research Council guidance

The UK Medical Research Council's widely used guidance for developing and evaluating complex interventions has been replaced by a new framework, commissioned jointly by the Medical Research Council and the National Institute for Health Research, which takes account of recent developments in theory and methods and the need to maximise the efficiency, use, and impact of research.

Ethical Frameworks and Global Health: A Narrative Review of the "Leave No One Behind" Principle

The "Leave No One Behind" (LNOB) principle, a fundamental commitment of the United Nations' Sustainable Development Goals, emphasizes the urgent need to address and reduce global health inequalities. As global health initiatives strive to uphold this principle, they face significant ethical challenges in balancing equity, resource allocation, and diverse health priorities. This narrative review critically examines these ethical dilemmas and their implications for translating LNOB into actionable global health strategies. A comprehensive literature search was conducted using PubMed, Scopus, Web of Science, and Semantic Scholar, covering publications from January 1990 to April 2024. The review included peer-reviewed articles, gray literature, and official reports that addressed the ethical dimensions of LNOB in global health contexts. A thematic analysis was employed to identify and synthesize recurring ethical issues, dilemmas, and proposed solutions. The thematic analysis identified 4 primary ethical tensions that complicate the operationalization of LNOB: (1) Universalism versus Targeting, where the challenge lies in balancing broad health improvements with targeted interventions for the most disadvantaged; (2) Resource Scarcity versus Equity; highlighting the ethical conflicts between maximizing efficiency and ensuring fairness; (3) Top-down versus Bottom-up Approaches, reflecting the tension between externally driven initiatives and local community needs; and (4) Short-term versus Long-term Sustainability, addressing the balance between immediate health interventions and sustainable systemic changes. To navigate these ethical challenges effectively, global health strategies must adopt a nuanced, context-sensitive approach incorporating structured decision-making processes and authentic community participation. The review advocates for systemic reforms that address the root causes of health disparities, promote equitable collaboration between health practitioners and marginalized communities, and align global health interventions with ethical imperatives. Such an approach is essential to truly operationalize the LNOB principle and foster sustainable health equity.

Preventing chronic malnutrition in children under 2 years in rural Angola (MuCCUA trial): protocol for the economic evaluation of a three-arm community cluster randomised controlled trial

INTRODUCTION

Chronic malnutrition is a serious problem in southern Angola with a prevalence of 49.9% and 37.2% in the provinces of Huila and Cunene, respectively. The MuCCUA (Mother and Child Chronic Undernutrition in Angola) trial is a community-based randomised controlled trial (c-RCT) which aims to evaluate the effectiveness of a nutrition supplementation plus standard of care intervention and a cash transfer plus standard of care intervention in preventing stunting, and to compare them with a standard of care alone intervention in southern Angola. This protocol describes the planned economic evaluation associated with the c-RCT.

METHODS AND ANALYSIS

We will conduct a cost-efficiency and cost-effectiveness analysis nested within the MuCCUA trial with a societal perspective, measuring programme, provider, participant and household costs. We will collect programme costs prospectively using a combined calculation method including quantitative and qualitative data. Financial costs will be estimated by applying activity-based costing methods to accounting records using time allocation sheets. We will estimate costs not included in accounting records by the ingredients approach, and indirect costs incurred by beneficiaries through interviews, household surveys and focus group discussions. Cost-efficiency will be estimated as cost per output achieved by combining activity-specific cost data with routine data on programme outputs. Cost-effectiveness will be assessed as cost per stunting case prevented. We will calculate incremental cost-effectiveness ratios comparing the additional cost per improved outcome of the different intervention arms and the standard of care. We will perform sensitivity analyses to assess robustness of results.

ETHICS AND DISSEMINATION

This economic evaluation will provide useful information to the Angolan Government and other policymakers on the most cost-effective intervention to prevent stunting in this and other comparable contexts. The protocol was approved by the República de Angola Ministério da Saúde Comité de Ética (27C.E/MINSA.INIS/2022). The findings of this study will be disseminated within academia and the wider policy sphere.

TRIAL REGISTRATION NUMBER

ClinicalTrials.gov Registry (NCT05571280).

Health Economics Research on Non-surgical Biomedical HIV Prevention: Identifying Gaps and Proposing a Way Forward

BACKGROUND AND OBJECTIVE

Although HIV prevention science has advanced over the last four decades, evidence suggests that prevention technologies do not always reach their full potential. Critical health economics evidence at appropriate decision-making junctures, particularly early in the development process, could help identify and address potential barriers to the eventual uptake of future HIV prevention products. This paper aims to identify key evidence gaps and propose health economics research priorities for the field of HIV non-surgical biomedical prevention.

METHODS

We used a mixed-methods approach with three distinct components: (i) three systematic literature reviews (costs and cost effectiveness, HIV transmission modelling and quantitative preference elicitation) to understand health economics evidence and gaps in the peer-reviewed literature; (ii) an online survey with researchers working in this field to capture gaps in yet-to-be published research (recently completed, ongoing and future); and (iii) a stakeholder meeting with key global and national players in HIV prevention, including experts in product development, health economics research and policy uptake, to uncover further gaps, as well as to elicit views on priorities and recommendations based on (i) and (ii).

RESULTS

Gaps in the scope of available health economics evidence were identified. Little research has been carried out on certain key populations (e.g. transgender people and people who inject drugs) and other vulnerable groups (e.g. pregnant people and people who breastfeed). Research is also lacking on preferences of community actors who often influence or enable access to health services among priority populations. Oral pre-exposure prophylaxis, which has been rolled out in many settings, has been studied in depth. However, research on newer promising technologies, such as long-acting pre-exposure prophylaxis formulations, broadly neutralising antibodies and multipurpose prevention technologies, is lacking. Interventions focussing on reducing intravenous and vertical transmission are also understudied. A disproportionate amount of evidence on low- and middle-income countries comes from two countries (South Africa and Kenya); evidence from other countries in sub-Saharan Africa as well as other low- and middle-income countries is needed. Further, data are needed on non-facility-based service delivery modalities, integrated service delivery and ancillary services. Key methodological gaps were also identified. An emphasis on equity and representation of heterogeneous populations was lacking. Research rarely acknowledged the complex and dynamic use of prevention technologies over time. Greater efforts are needed to collect primary data, quantify uncertainty, systematically compare the full range of prevention options available, and validate pilot and modelling data once interventions are scaled up. Clarity on appropriate cost-effectiveness outcome measures and thresholds is also lacking. Lastly, research often fails to reflect policy-relevant questions and approaches.

CONCLUSIONS

Despite a large body of health economics evidence on non-surgical biomedical HIV prevention technologies, important gaps in the scope of evidence and methodology remain. To ensure that high-quality research influences key decision-making junctures and facilitates the delivery of prevention products in a way that maximises impact, we make five broad recommendations related to: improved study design, an increased focus on service delivery, greater community and stakeholder engagement, the fostering of an active network of partners across sectors and an enhanced application of research.

How is the Societal Perspective Defined in Health Technology Assessment? Guidelines from Around the Globe

Some researchers have argued that the aim of an economic evaluation should be to offer guidance on resource allocation based on public interest from a societal perspective. The application of a societal perspective in health technology assessment (HTA), while common in many published studies, is not mandated in most countries, and there is limited discussion on what the societal perspective should encompass. This study aimed to systematically compare and contrast the HTA guidelines in different countries. HTA methods guidelines were identified through international HTA networks, such as the Professional Society for Health Economics and Outcomes Research (ISPOR) and Guide to Economic Analysis Research (GEAR). The respective HTA agencies were grouped into two categories: well-established and newly developed, based on the establishment date. Data extracted from the guidelines summarised the methodological details in the reference cases, including specifics on the societal perspective. The database search yielded 46 guidelines, and 65% explicitly considered the societal perspective. The maturity of these agencies is reflected in their attitudes towards the societal perspective; the societal perspective is defined in 73% of the guidelines of well-established agencies and only 56% of those of newly developed agencies. The guidelines from multipayer healthcare systems are more likely to consider the societal perspective. Although most guidelines from the well-established agencies recommend the inclusion of a societal perspective, the types of costs and consequences that should be included and the recommended approaches to valuing them are variable. The direct costs to family and carers were included in 73% of the societal perspective definitions, while non-health outcomes were considered in only 40%. Most HTA guidelines lack clear guidance on what to include under specific perspectives. Considering the recent advancements in economic evaluation methods, it is timely to rethink the role of the societal perspective in HTA guidelines and adopt a more comprehensive perspective to include all costs and consequences of healthcare services.

Are intersectoral costs considered in economic evaluations of interventions relating to sexually transmitted infections (STIs)? A systematic review

BACKGROUND/OBJECTIVE

Sexually transmitted infections (STIs) not only have an impact on the health sector but also the private resources of those affected, their families and other sectors of society (i.e. labour, education). This study aimed to i) review and identify economic evaluations of interventions relating to STIs, which aimed to include a societal perspective; ii) analyse the intersectoral costs (i.e. costs broader than healthcare) included; iii) categorise these costs by sector; and iv) assess the impact of intersectoral costs on the overall study results.

METHODS

Seven databases were searched: MEDLINE (PubMed), EMBASE (Ovid), Web of Science, CINAHL, PsycINFO, EconLit and NHS EED. Key search terms included terms for economic evaluation, STIs and specific infections. This review considered trial- and model-based economic evaluations conducted in an OECD member country. Studies were included that assessed intersectoral costs. Intersectoral costs were extracted and categorised by sector using Drummond's cost classification scheme (i.e. patient/family, productivity, costs in other sectors). A narrative synthesis was performed.

RESULTS

Twenty-nine studies qualified for data extraction and narrative synthesis. Twenty-eight studies applied a societal perspective of which 8 additionally adopted a healthcare or payer perspective, or both. One study used a modified payer perspective. The following sectors were identified: patient/family, informal care, paid labour (productivity), non-paid opportunity costs, education, and consumption. Patient/family costs were captured in 11 studies and included patient time, travel expenses, out-of-pocket costs and premature burial costs. Informal caregiver support (non-family) and unpaid help by family/friends was captured in three studies. Paid labour losses were assessed in all but three studies. Three studies also captured the costs and inability to perform non-paid work. Educational costs and future non-health consumption costs were each captured in one study. The inclusion of intersectoral costs resulted in more favourable cost estimates.

CONCLUSIONS

This systematic review suggests that economic evaluations of interventions relating to STIs that adopt a societal perspective tend to be limited in scope. There is an urgent need for economic evaluations to be more comprehensive in order to allow policy/decision-makers to make better-informed decisions.

The ethics of aggregation in cost-effectiveness analysis or, "on books, bookshelves, and budget impact"

In deciding how to allocate resources, healthcare priority-setters are increasingly paying attention to an intervention's budget impact alongside its cost-effectiveness. Some argue that approaches that use budget impact as a substantive consideration unfairly disadvantage individuals who belong to large patient groups. Others reject such claims of "numerical discrimination" on the grounds that consideration of the full budget impact of an intervention's adoption is necessary to properly estimate opportunity cost. This paper summarizes this debate and advances a new argument against modifying the cost-effectiveness threshold used for decision-making based on a technology's anticipated budget impact. In making this argument, the paper sets out how the apparent link between budget impact and opportunity cost is largely broken if the effects of a technology's adoption are disaggregated, while highlighting that the theoretical aggregation of effects during cost-effectiveness analysis likely only poorly reflects the operation of the health system in practice. As such, it identifies a need for healthcare priority-setters to be cognizant of the ethical implications associated with aggregating the effects of a technology's adoption for the purpose of decision-making. Throughout the paper, these arguments are illustrated with reference to a "bookshelf" analogy borrowed from previous work.

Supply-Side Cost-Effectiveness Thresholds: Questions for Evidence-Based Policy

There is growing interest in cost-effectiveness thresholds as a tool to inform resource allocation decisions in health care. Studies from several countries have sought to estimate health system opportunity costs, which supply-side cost-effectiveness thresholds are intended to represent. In this paper, we consider the role of empirical estimates of supply-side thresholds in policy-making. Recent studies estimate the cost per unit of health based on average displacement or outcome elasticity. We distinguish the types of point estimates reported in empirical work, including marginal productivity, average displacement, and outcome elasticity. Using this classification, we summarise the limitations of current approaches to threshold estimation in terms of theory, methods, and data. We highlight the questions that arise from alternative interpretations of thresholds and provide recommendations to policymakers seeking to use a supply-side threshold where the evidence base is emerging or incomplete. We recommend that: (1) policymakers must clearly define the scope of the application of a threshold, and the theoretical basis for empirical estimates should be consistent with that scope; (2) a process for the assessment of new evidence and for determining changes in the threshold to be applied in policy-making should be created; (3) decision-making processes should retain flexibility in the application of a threshold; and (4) policymakers should provide support for decision-makers relating to the use of thresholds and the implementation of decisions stemming from their application.

Could a Shigella vaccine impact long-term health outcomes?: Summary report of an expert meeting to inform a Shigella vaccine public health value proposition, March 24 and 29, 2021

Shigellosis is a leading cause of diarrhea and dysentery in young children from low to middle-income countries and adults experiencing traveler’s diarrhea worldwide. In addition to acute illness, infection by Shigella bacteria is associated with stunted growth among children, which has been linked to detrimental long-term health, developmental, and economic outcomes. On March 24 and 29, 2021, PATH convened an expert panel to discuss the potential impact of Shigella vaccines on these long-term outcomes. Based on current empirical evidence, this discussion focused on whether Shigella vaccines could potentially alleviate the long-term burden associated with Shigella infections. Also, the experts provided recommendations about how to best model the burden, health and vaccine impact, and economic consequences of Shigella infections. This international multidisciplinary panel included 13 scientists, physicians, and economists from multiple relevant specialties.

According to the panel, while the relationship between Shigella infections and childhood growth deficits is complex, this relationship likely exists. Vaccine probe studies are the crucial next step to determine whether vaccination could ameliorate Shigella infection-related long-term impacts. Infants should be vaccinated during their first year of life to maximize their protection from severe acute health outcomes and ideally reduce stunting risk and subsequent negative long-term developmental and health impacts. With vaccine schedule crowding, targeted or combination vaccination approaches would likely increase vaccine uptake in high-burden areas. Shigella impact and economic assessment models should include a wider range of linear growth outcomes. Also, these models should produce a spectrum of results—ones addressing immediate benefits for usual health care decision-makers and others that include broader health impacts, providing a more comprehensive picture of vaccination benefits. While many of the underlying mechanisms of this relationship need better characterization, the remaining gaps can be best addressed by collecting data post-vaccine introduction or through large trials.

Economic evaluation of psychological treatments for common mental disorders in low- and middle-income countries: a systematic review

Common mental disorders (CMDs) constitute a major public health and economic burden on low- and middle-income countries (LMICs). Systematic reviews of economic evaluations of psychological treatments for CMDs are limited. This systematic review examines methods, reports findings and appraises the quality of economic evaluations of psychological treatments for CMDs in LMICs. We searched a range of bibliographic databases (including PubMed, EconLit, APA-PsycINFO and Cochrane library) and the African Journals Online (AJoL) and Google Scholar platforms. We used a pre-populated template to extract data and the Drummond & Jefferson checklist for quality appraisal. We present results as a narrative synthesis. The review included 26 studies, mostly from Asia (12) and Africa (9). The majority were cost-effectiveness analyses (12), some were cost-utility analyses (5), with one cost-benefit analysis or combinations of economic evaluations (8). Most interventions were considered either cost-effective or potentially cost-effective (22), with 3 interventions being not cost-effective. Limitations were noted regarding appropriateness of conclusions drawn on cost-effectiveness, the use of cost-effectiveness thresholds and application of 'societal' incremental cost-effectiveness ratios to reflect value for money (VfM) of treatments. Non-specialist health workers (NSHWs) delivered most of the treatments (16) for low-cost delivery at scale, and costs should reflect the true opportunity cost of NSHWs' time to support the development of a sustainable cadre of health care providers. There is a 4-fold increase in economic evaluations of CMD psychological treatments in the last decade over the previous one. Yet, findings from this review highlight the need for better application of economic evaluation methodology to support resource allocation towards the World Health Organization recommended first-line treatments of CMDs. We suggest impact inventories to capture societal economic gains and propose a VfM assessment framework to guide researchers in evaluating cost-effectiveness.

Measurement of benefits in economic evaluations of nutrition interventions in low- and middle-income countries: A systematic review

Economic evaluation of nutrition interventions that compares the costs to benefits is essential to priority-setting. However, there are unique challenges to synthesizing the findings of multi-sectoral nutrition interventions due to the diversity of potential benefits and the methodological differences among sectors in measuring them. This systematic review summarises literature on the interventions, sectors, benefit terminology and benefit types included in cost-effectiveness, cost-utility and benefit-cost analyses (CEA, CUA and BCA, respectively) of nutrition interventions in low- and middle-income countries. A systematic search of five databases published from January 2010 to September 2019 with expert consultation yielded 2794 studies, of which 93 met all inclusion criteria. Eighty-seven per cent of the included studies included interventions delivered from only one sector, with almost half from the health sector (43%), followed by food/agriculture (27%), water, sanitation and hygiene (WASH) (10%), and social protection (8%). Only 9% of studies assessed programmes involving more than one sector (health, food/agriculture, social protection and/or WASH). Eighty-one per cent of studies used more than one term to refer to intervention benefits. The included studies calculated 128 economic evaluation ratios (57 CEAs, 39 CUAs and 32 BCAs), and the benefits they included varied by sector. Nearly 60% measured a single benefit category, most frequently nutritional status improvements; other health benefits, cognitive/education gains, dietary diversity, food security, knowledge/attitudes/practices and income were included in less than 10% of all ratios. Additional economic evaluation of non-health and multi-sector interventions, and incorporation of benefits beyond nutritional improvements (including cost savings) in future economic evaluations is recommended.

Prevention of violence against women and girls: A cost-effectiveness study across 6 low- and middle-income countries

BACKGROUND

Violence against women and girls (VAWG) is a human rights violation with social, economic, and health consequences for survivors, perpetrators, and society. Robust evidence on economic, social, and health impact, plus the cost of delivery of VAWG prevention, is critical to making the case for investment, particularly in low- and middle-income countries (LMICs) where health sector resources are highly constrained. We report on the costs and health impact of VAWG prevention in 6 countries.

METHODS AND FINDINGS

We conducted a trial-based cost-effectiveness analysis of VAWG prevention interventions using primary data from 5 randomised controlled trials (RCTs) in sub-Saharan Africa and 1 in South Asia. We evaluated 2 school-based interventions aimed at adolescents (11 to 14 years old) and 2 workshop-based (small group or one to one) interventions, 1 community-based intervention, and 1 combined small group and community-based programme all aimed at adult men and women (18+ years old). All interventions were delivered between 2015 and 2018 and were compared to a do-nothing scenario, except for one of the school-based interventions (government-mandated programme) and for the combined intervention (access to financial services in small groups). We computed the health burden from VAWG with disability-adjusted life year (DALY). We estimated per capita DALYs averted using statistical models that reflect each trial's design and any baseline imbalances. We report cost-effectiveness as cost per DALY averted and characterise uncertainty in the estimates with probabilistic sensitivity analysis (PSA) and cost-effectiveness acceptability curves (CEACs), which show the probability of cost-effectiveness at different thresholds. We report a subgroup analysis of the small group component of the combined intervention and no other subgroup analysis. We also report an impact inventory to illustrate interventions' socioeconomic impact beyond health. We use a 3% discount rate for investment costs and a 1-year time horizon, assuming no effects post the intervention period. From a health sector perspective, the cost per DALY averted varies between US$222 (2018), for an established gender attitudes and harmful social norms change community-based intervention in Ghana, to US$17,548 (2018) for a livelihoods intervention in South Africa. Taking a societal perspective and including wider economic impact improves the cost-effectiveness of some interventions but reduces others. For example, interventions with positive economic impacts, often those with explicit economic goals, offset implementation costs and achieve more favourable cost-effectiveness ratios. Results are robust to sensitivity analyses. Our DALYs include a subset of the health consequences of VAWG exposure; we assume no mortality impact from any of the health consequences included in the DALYs calculations. In both cases, we may be underestimating overall health impact. We also do not report on participants' health costs.

CONCLUSIONS

We demonstrate that investment in established community-based VAWG prevention interventions can improve population health in LMICs, even within highly constrained health budgets. However, several VAWG prevention interventions require further modification to achieve affordability and cost-effectiveness at scale. Broadening the range of social, health, and economic outcomes captured in future cost-effectiveness assessments remains critical to justifying the investment urgently required to prevent VAWG globally.

Informing decisions with disparate stakeholders: cross-sector evaluation of cash transfers in Malawi

The Social Cash Transfer Programme (SCTP) in Malawi is a cross-sectoral policy with impacts on health, education, nutrition, agriculture and welfare. Implementation of the SCTP requires collaboration across sectors and across national and international stakeholders. Economic evaluation can inform investment by indicating whether benefits exceed costs, but economic evaluations that provide an overall benefit-cost ratio typically assume a common agreed objective and agreed set of value judgements. In reality, the various stakeholders involved in the delivery of the SCTP may have different remits and objectives and may differ in how they value the impacts of the programme. We use the SCTP as a case study to illustrate a cross-sectoral analytical framework that accounts for these differences. The stakeholders that contribute to the SCTP include the Ministry of Gender, Ministry of Finance, Ministry of Economic Planning and Development and Global Fund. We estimate how the SCTP changes outcomes in education, health, net production and poverty, and distinguish outcomes in three groups: SCTP recipients; population in Malawi not eligible for the SCTP and population in other countries. After estimating the direct effects and opportunity costs from investing in the SCTP, we summarize the results according to different perspectives. The SCTP is estimated to provide benefits in excess of costs from the perspective of national stakeholders. From the perspective of an international donor interested in health outcomes, its health benefits do not outweigh the opportunity costs unless health improvement in SCTP recipients is valued at 18 times that of other potential spending beneficiaries or the donor values broader outcomes than health alone. This work illustrates the potential of a cross-sectoral economic evaluation to guide debate about stakeholder contributions to the SCTP, and the value judgements required to favour the SCTP above other policy options.

Economic Evaluation of Public Health Interventions: An Application to Interventions for the Prevention of Violence Against Women and Girls Implemented by the "What Works to Prevent Violence Against Women and Girls?" Global Program

Violence against women and girls (VAWG) has important social, economic, and public health impacts. Governments and international donors are increasing their investment in VAWG prevention programs, yet clear guidelines to assess the "value for money" of these interventions are lacking. Improved costing and economic evaluation of VAWG prevention can support programming through supporting priority setting, justifying investment, and planning the financing of VAWG prevention services. This article sets out a standardized methodology for the economic evaluation of complex, that is, multicomponent and/or multiplatform, programs designed to prevent VAWG in low- and middle-income countries (LMICs). It outlines an approach that can be used alongside the most recent guidance for the economic evaluation of public health interventions in LMICs. It defines standardized methods of data collection and analysis, outcomes, and unit costs (i.e., average costs per person reached, output or service delivered), and provides guidance to investigate the uncertainty in cost-effectiveness estimates and report results. The costing approach has been developed and piloted as part of the "What Works to Prevent Violence Against Women and Girls?" (What Works?) program in five countries. This article and its supplementary material can be used by both economists and non-economists to contribute to the generation of new cost-effectiveness data on VAWG prevention, and ultimately improve the allocative efficiency and financing across VAWG programs.

Intersectoral costs of sexually transmitted infections (STIs) and HIV: a systematic review of cost-of-illness (COI) studies

Background

Sexually transmitted infections (STIs) and HIV can generate costs both within and outside the health sector (i.e. intersectoral costs). This systematic review aims (i) to explore the intersectoral costs associated with STIs and HIV considered in cost-of-illness (COI) studies, (ii) to categorise and analyse these costs according to cost sectors, and (iii) to illustrate the impact of intersectoral costs on the total cost burden.

Methods

Medline (PubMed), EMBASE (Ovid), Web of Science, CINAHL, PsycINFO, EconLit and NHS EED were searched between 2009 and 2019. Key search terms included terms for cost-of-illness, cost analysis and all terms for STIs including specific infections. Studies were included that assessed intersectoral costs. A standardised data extraction form was adopted. A cost component table was established based on pre-defined sector-specific classification schemes. Cost results for intersectoral costs were recorded. The quality of studies was assessed using a modified version of the CHEC-list.

Results

75 COI studies were considered for title/abstract screening. Only six studies were available in full-text and eligible for data extraction and narrative synthesis. Intersectoral costs were captured in the following sectors: Patient & family, Informal care and Productivity (Paid Labour). Patient & family costs were addressed in four studies, including patient out-of-pocket payments/co-payments and travel costs. Informal care costs including unpaid (home) care support by family/friends and other caregiver costs were considered in three studies. All six studies estimated productivity costs for paid labour including costs in terms of absenteeism, disability, cease-to-work, presenteeism and premature death. Intersectoral costs largely contributed to the total economic cost burden of STIs and HIV. The quality assessment revealed methodological differences.

Conclusions

It is evident that intersectoral costs associated with STIs and HIV are substantial. If relevant intersectoral costs are not included in cost analyses the total cost burden of STIs and HIV to society is severely underestimated. Therefore, intersectoral costs need to be addressed in order to ensure the total economic burden of STIs and HIV on society is assessed, and communicated to policy/decision-makers.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-021-07147-z.

A Systematic Review of Demand-Side Methods of Estimating the Societal Monetary Value of Health Gain

OBJECTIVES

Although many reviews of the literature on cost-effectiveness thresholds (CETs) exist, the availability of new studies and the absence of a fully comprehensive analysis warrant a new review. This study systematically reviews demand-side methods for estimating the societal monetary value of health gain.

METHODS

Several electronic databases were searched from inception to October 2019. To be included, a study had to be an original article in any language, with a clearly described method for estimating the societal monetary values of health gain and with all estimated values reported. Estimates were converted to US dollars ($), using purchasing power parity (PPP) exchange rates and the gross domestic product (GDP) per capita (2019).

RESULTS

We included 53 studies; 45 used direct approach and 8 used indirect approach. Median estimates from the direct approach were PPP$ 24 942 (range 554-1 301 912) per quality-adjusted life-year (QALY), which were typically 0.53 (range 0.02-24.08) GDP per capita. Median estimates using the indirect approach were PPP$ 310 051 (range 36 402-7 574 870) per QALY, which accounted for 7.87 (range 0.68-116.95) GDP per capita.

CONCLUSIONS

Our review found that the societal values of health gain or CETs were less than GDP per capita. The great variety in methods and estimates suggests that a more standardized and internationally agreed methodology for estimating CET is warranted. Multiple CETs may have a role when QALYs are not equally valued from a societal perspective (eg, QALYs accruing to people near death compared with equivalent QALYs to others).

Framework for the development and evaluation of complex interventions: gap analysis, workshop and consultation-informed update

BACKGROUND

The Medical Research Council published the second edition of its framework in 2006 on developing and evaluating complex interventions. Since then, there have been considerable developments in the field of complex intervention research. The objective of this project was to update the framework in the light of these developments. The framework aims to help research teams prioritise research questions and design, and conduct research with an appropriate choice of methods, rather than to provide detailed guidance on the use of specific methods.

METHODS

There were four stages to the update: (1) gap analysis to identify developments in the methods and practice since the previous framework was published; (2) an expert workshop of 36 participants to discuss the topics identified in the gap analysis; (3) an open consultation process to seek comments on a first draft of the new framework; and (4) findings from the previous stages were used to redraft the framework, and final expert review was obtained. The process was overseen by a Scientific Advisory Group representing the range of relevant National Institute for Health Research and Medical Research Council research investments.

RESULTS

Key changes to the previous framework include (1) an updated definition of complex interventions, highlighting the dynamic relationship between the intervention and its context; (2) an emphasis on the use of diverse research perspectives: efficacy, effectiveness, theory-based and systems perspectives; (3) a focus on the usefulness of evidence as the basis for determining research perspective and questions; (4) an increased focus on interventions developed outside research teams, for example changes in policy or health services delivery; and (5) the identification of six 'core elements' that should guide all phases of complex intervention research: consider context; develop, refine and test programme theory; engage stakeholders; identify key uncertainties; refine the intervention; and economic considerations. We divide the research process into four phases: development, feasibility, evaluation and implementation. For each phase we provide a concise summary of recent developments, key points to address and signposts to further reading. We also present case studies to illustrate the points being made throughout.

LIMITATIONS

The framework aims to help research teams prioritise research questions and design and conduct research with an appropriate choice of methods, rather than to provide detailed guidance on the use of specific methods. In many of the areas of innovation that we highlight, such as the use of systems approaches, there are still only a few practical examples. We refer to more specific and detailed guidance where available and note where promising approaches require further development.

CONCLUSIONS

This new framework incorporates developments in complex intervention research published since the previous edition was written in 2006. As well as taking account of established practice and recent refinements, we draw attention to new approaches and place greater emphasis on economic considerations in complex intervention research. We have introduced a new emphasis on the importance of context and the value of understanding interventions as 'events in systems' that produce effects through interactions with features of the contexts in which they are implemented. The framework adopts a pluralist approach, encouraging researchers and research funders to adopt diverse research perspectives and to select research questions and methods pragmatically, with the aim of providing evidence that is useful to decision-makers.

FUTURE WORK

We call for further work to develop relevant methods and provide examples in practice. The use of this framework should be monitored and the move should be made to a more fluid resource in the future, for example a web-based format that can be frequently updated to incorporate new material and links to emerging resources.

FUNDING

This project was jointly funded by the Medical Research Council (MRC) and the National Institute for Health Research (Department of Health and Social Care 73514).

Economic evaluation of an early childhood development center–based agriculture and nutrition intervention in Malawi

Malnutrition is a leading cause of death and disability among children in low-income countries. Nutrition-sensitive interventions show promise in increasing food access and improving diets. There are possible synergies of integrating these programs with other sectors, improving effectiveness by leveraging resources. However, economic evaluations of these multi-sectoral programs are limited. We aimed to estimate the cost efficiency, cost-effectiveness, benefit-cost ratio, and net benefit of using community-based early childhood development (ECD) centers as platforms for an intervention promoting agricultural production and nutrition among households with young children in Malawi. The intervention was costed using bottom-up micro-costing and top-down expenditure analysis with a societal perspective and a 12-month horizon. Effectiveness estimates were derived from a cluster-randomized control trial. Premature deaths and stunting cases averted were estimated using the Lived Saved Tool. We calculated DALYs averted, and the value of three benefits streams resulting from reductions in premature mortality, increases in lifetime productivity and household agricultural productivity. We transferred the US value of a statistical life (VSL) to Malawi using an income elasticity of 1.5, and a 10% discount rate. Probabilistic sensitivity analysis was conducted using a Monte Carlo model. The intervention cost $197,377, reaching 4,806 beneficiaries at $41 per beneficiary, $595 per case of stunting, $18,310 per death, and $516 per DALY averted. Net benefit estimates ranged from $507,589 to $4,678,258, and benefit-cost ratios from 3.57 to 24.70. Sensitivity analyses confirmed a positive return on investment. Implementing agriculture-nutrition interventions through ECD platforms may be an efficient use of resources in Malawi and similar contexts.

Early intervention, treatment and rehabilitation of employees with common mental disorders by using psychotherapeutic consultation at work: study protocol of a randomised controlled multicentre trial (friaa project)

BACKGROUND

Common mental disorders are one of the leading causes for sickness absence and early retirement due to reduced health. Furthermore, a treatment gap for common mental disorders has been described worldwide. Within this study, psychotherapeutic consultation at work defined as a tailored, module-based and work-related psychotherapeutic intervention will be applied to improve mental health care.

METHODS

This study comprises a randomised controlled multicentre trial with 1:1 allocation to an intervention and control group. In total, 520 employees with common mental disorders shall be recruited from companies being located around five study centres in Germany. Besides care as usual, the intervention group will receive up to 17 sessions of psychotherapy. The first session will include basics diagnostics and medical indication of treatment and the second session will include work-related diagnostics. Then, participants of the intervention group may receive work-related psychotherapeutic consultation for up to ten sessions. Further psychotherapeutic consultation during return to work for up to five sessions will be offered where appropriate. The control group will receive care as usual and the first intervention session of basic diagnostics and medical indication of treatment. After enrolment to the study, participants will be followed up after nine (first follow-up) and fifteen (second follow-up) months. Self-reported days of sickness absence within the last 6 months at the second follow-up will be used as the primary outcome and self-efficacy at the second follow-up as the secondary outcome. Furthermore, a cost-benefit assessment related to costs of common mental disorders for social insurances and companies will be performed.

DISCUSSION

Psychotherapeutic consultation at work represents a low threshold care model aiming to overcome treatment gaps for employees with common mental disorders. If successfully implemented and evaluated, it might serve as a role model to the care of employees with common mental disorders and might be adopted in standard care in cooperation with sickness and pension insurances in Germany.

TRIAL REGISTRATION

The friaa project was registered at the German Clinical Trial Register (DRKS) at 01.03.2021 (DRKS00023049): https://www.drks.de/drks_web/navigate.do?navigationId=trial.HTML&TRIAL_ID=DRKS00023049 .

Strengthening health systems to improve the value of tuberculosis diagnostics in South Africa: A cost and cost-effectiveness analysis

BACKGROUND

In South Africa, replacing smear microscopy with Xpert-MTB/RIF (Xpert) for tuberculosis diagnosis did not reduce mortality and was cost-neutral. The unchanged mortality has been attributed to suboptimal Xpert implementation. We developed a mathematical model to explore how complementary investments may improve cost-effectiveness of the tuberculosis diagnostic algorithm.

METHODS

Complementary investments in the tuberculosis diagnostic pathway were compared to the status quo. Investment scenarios following an initial Xpert test included actions to reduce pre-treatment loss-to-follow-up; supporting same-day clinical diagnosis of tuberculosis after a negative result; and improving access to further tuberculosis diagnostic tests following a negative result. We estimated costs, deaths and disability-adjusted-life-years (DALYs) averted from provider and societal perspectives. Sensitivity analyses explored the mediating influence of behavioural, disease- and organisational characteristics on investment effectiveness.

FINDINGS

Among a cohort of symptomatic patients tested for tuberculosis, with an estimated active tuberculosis prevalence of 13%, reducing pre-treatment loss-to-follow-up from ~20% to ~0% led to a 4% (uncertainty interval [UI] 3; 4%) reduction in mortality compared to the Xpert scenario. Improving access to further tuberculosis diagnostic tests from ~4% to 90% among those with an initial negative Xpert result reduced overall mortality by 28% (UI 27; 28) at $39.70/ DALY averted. Effectiveness of investment scenarios to improve access to further diagnostic tests was dependent on a high return rate for follow-up visits.

INTERPRETATION

Investing in direct and indirect costs to support the TB diagnostic pathway is potentially highly cost-effective.

Cost-effectiveness analysis of public health interventions with impacts on health and criminal justice: An applied cross-sectoral analysis of an alcohol misuse intervention

Cost-effectiveness analyses of health care programs often focus on maximizing health and ignore nonhealth impacts. Assessing the cost-effectiveness of public health interventions from a narrow health care perspective would likely underestimate their full impact, and potentially lead to inefficient decisions about funding. The aim of this study is to provide a practical application of a recently proposed framework for the economic evaluation of public health interventions, evaluating an intervention to reduce alcohol misuse in criminal offenders. This cross-sectoral analysis distinguishes benefits and opportunity costs for different sectors, makes explicit the value judgments required to consider alternative perspectives, and can inform heterogeneous decision makers with different objectives in a transparent manner. Three interventions of increasing intensity are compared: client information leaflet, brief advice, and brief lifestyle counseling. Health outcomes are measured in quality-adjusted life-years and criminal justice outcomes in reconvictions. Costs considered include intervention costs, costs to the NHS and costs to the criminal justice system. The results are presented for four different perspectives: "narrow" health care perspective; criminal justice system perspective; "full" health care perspective; and joint "full" health and criminal justice perspective. Conclusions and recommendations differ according to the normative judgment on the appropriate perspective for the evaluation.

From impact evaluation to decision-analysis: assessing the extent and quality of evidence on ‘value for money’ in health impact evaluations in low- and middle-income countries

Background: Health impact evaluations (HIEs) are currently the main way of assessing policy changes in low-and middle-income countries (LMICs). However, evidence on effectiveness alone cannot reliably inform decisions over the allocation of limited resources. Health economic evaluation provides a suitable framework for ‘value for money’ assessments. Methods: In this article we explore to what extent economic evaluations have been conducted alongside published health impact evaluations, then we assess the quality of these, using criteria from an economic evaluation reference case developed for use in LMICs. Results: Among the 2419 HIEs stored in the International Initiative for Impact Evaluations (3ie) database, and among the 8155 studies identified by the Ovid Medline database search, only 70 studies included an economic evaluation. When measured against the quality assessment criteria, study quality showed great variation. Many studies did not fulfil the basic requirements for economic evaluation, such as stating the perspective of the budget holder, using generic health measures that can be compared across diseases, or suitably reflecting uncertainty. Conclusions: Greater effort should be directed towards bringing the fields of impact evaluation and economic evaluation together to better inform resource allocation decisions in global health.

Financing intersectoral action for health: a systematic review of co-financing models

BACKGROUND

Addressing the social and other non-biological determinants of health largely depends on policies and programmes implemented outside the health sector. While there is growing evidence on the effectiveness of interventions that tackle these upstream determinants, the health sector does not typically prioritise them. From a health perspective, they may not be cost-effective because their non-health outcomes tend to be ignored. Non-health sectors may, in turn, undervalue interventions with important co-benefits for population health, given their focus on their own sectoral objectives. The societal value of win-win interventions with impacts on multiple development goals may, therefore, be under-valued and under-resourced, as a result of siloed resource allocation mechanisms. Pooling budgets across sectors could ensure the total multi-sectoral value of these interventions is captured, and sectors' shared goals are achieved more efficiently. Under such a co-financing approach, the cost of interventions with multi-sectoral outcomes would be shared by benefiting sectors, stimulating mutually beneficial cross-sectoral investments. Leveraging funding in other sectors could off-set flat-lining global development assistance for health and optimise public spending. Although there have been experiments with such cross-sectoral co-financing in several settings, there has been limited analysis to examine these models, their performance and their institutional feasibility.

AIM

This study aimed to identify and characterise cross-sectoral co-financing models, their operational modalities, effectiveness, and institutional enablers and barriers.

METHODS

We conducted a systematic review of peer-reviewed and grey literature, following PRISMA guidelines. Studies were included if data was provided on interventions funded across two or more sectors, or multiple budgets. Extracted data were categorised and qualitatively coded.

RESULTS

Of 2751 publications screened, 81 cases of co-financing were identified. Most were from high-income countries (93%), but six innovative models were found in Uganda, Brazil, El Salvador, Mozambique, Zambia, and Kenya that also included non-public and international payers. The highest number of cases involved the health (93%), social care (64%) and education (22%) sectors. Co-financing models were most often implemented with the intention of integrating services across sectors for defined target populations, although models were also found aimed at health promotion activities outside the health sector and cross-sectoral financial rewards. Interventions were either implemented and governed by a single sector or delivered in an integrated manner with cross-sectoral accountability. Resource constraints and political relevance emerged as key enablers of co-financing, while lack of clarity around the roles of different sectoral players and the objectives of the pooling were found to be barriers to success. Although rigorous impact or economic evaluations were scarce, positive process measures were frequently reported with some evidence suggesting co-financing contributed to improved outcomes.

CONCLUSION

Co-financing remains in an exploratory phase, with diverse models having been implemented across sectors and settings. By incentivising intersectoral action on structural inequities and barriers to health interventions, such a novel financing mechanism could contribute to more effective engagement of non-health sectors; to efficiency gains in the financing of universal health coverage; and to simultaneously achieving health and other well-being related sustainable development goals.

An Educational Review About Using Cost Data for the Purpose of Cost-Effectiveness Analysis

This paper provides an educational review covering the consideration of costs for cost-effectiveness analysis (CEA), summarising relevant methods and research from the published literature. Cost data are typically generated by applying appropriate unit costs to healthcare resource-use data for patients. Trial-based evaluations and decision analytic modelling represent the two main vehicles for CEA. The costs to consider will depend on the perspective taken, with conflicting recommendations ranging from focusing solely on healthcare to the broader 'societal' perspective. Alternative sources of resource-use are available, including medical records and forms completed by researchers or patients. Different methods are available for the statistical analysis of cost data, although consideration needs to be given to the appropriate methods, given cost data are typically non-normal with a mass point at zero and a long right-hand tail. The choice of covariates for inclusion in econometric models also needs careful consideration, focusing on those that are influential and that will improve balance and precision. Where data are missing, it is important to consider the type of missingness and then apply appropriate analytical methods, such as imputation. Uncertainty around costs should also be reflected to allow for consideration on the impacts of the CEA results on decision uncertainty. Costs should be discounted to account for differential timing, and are typically inflated to a common cost year. The choice of methods and sources of information used when accounting for cost information within CEA will have an effect on the subsequent cost-effectiveness results and how information is presented to decision makers. It is important that the most appropriate methods are used as overlooking the complicated nature of cost data could lead to inaccurate information being given to decision makers.

Health, financial, and education gains of investing in preventive chemotherapy for schistosomiasis, soil-transmitted helminthiases, and lymphatic filariasis in Madagascar: A modeling study

BACKGROUND

Neglected tropical diseases (NTDs) account for a large disease burden in sub-Saharan Africa. While the general cost-effectiveness of NTD interventions to improve health outcomes has been assessed, few studies have also accounted for the financial and education gains of investing in NTD control.

METHODS

We built on extended cost-effectiveness analysis (ECEA) methods to assess the health gains (e.g. infections, disability-adjusted life years or DALYs averted), household financial gains (out-of-pocket expenditures averted), and education gains (cases of school absenteeism averted) for five NTD interventions that the government of Madagascar aims to roll out nationally. The five NTDs considered were schistosomiasis, lymphatic filariasis, and three soil-transmitted helminthiases (Ascaris lumbricoides, Trichuris trichiura, and hookworm infections).

RESULTS

The estimated incremental cost-effectiveness for the roll-out of preventive chemotherapy for all NTDs jointly was USD125 per DALY averted (95% uncertainty range: 65-231), and its benefit-cost ratio could vary between 5 and 31. Our analysis estimated that, per dollar spent, schistosomiasis preventive chemotherapy, in particular, could avert a large number of infections (176,000 infections averted per $100,000 spent), DALYs (2,000 averted per $100,000 spent), and cases of school absenteeism (27,000 school years gained per $100,000 spent).

CONCLUSION

This analysis incorporates financial and education gains into the economic evaluation of health interventions, and therefore provides information about the efficiency of attainment of three Sustainable Development Goals (SDGs). Our findings reveal how the national scale-up of NTD control in Madagascar can help address health (SDG3), economic (SDG1), and education (SDG4) goals. This study further highlights the potentially large societal benefits of investing in NTD control in low-resource settings.

Primary and secondary prevention interventions for cardiovascular disease in low-income and middle-income countries: a systematic review of economic evaluations

Background

Cardiovascular disease (CVD) is the leading cause of deaths globally, with greatest premature mortality in the low- and middle-income countries (LMIC). Many of these countries, especially in sub-Saharan Africa, have significant budget constraints. The need for current evidence on which interventions offer good value for money to stem this CVD epidemic motivates this study.

Methods

In this systematic review, we included studies reporting full economic evaluations of individual and population-based interventions (pharmacologic and non-pharmacologic), for primary and secondary prevention of CVD among adults in LMIC. Several medical (PubMed, EMBASE, SCOPUS, Web of Science) and economic (EconLit, NHS EED) databases and grey literature were searched. Screening of studies and data extraction was done independently by two reviewers. Drummond’s checklist and the National Institute for Health and Care Excellence quality rating scale were used in the quality appraisal for all studies used to inform this evidence synthesis.

Results

From a pool of 4059 records, 94 full texts were read and 50 studies, which met our inclusion criteria, were retained for our narrative synthesis. Most of the studies were from middle-income countries and predominantly of high quality. The majority were modelled evaluations, and there was significant heterogeneity in methods. Primary prevention studies dominated secondary prevention. Most of the economic evaluations were performed for pharmacological interventions focusing on blood pressure, cholesterol lowering and antiplatelet aggregants. The greatest majority were cost-effective. Compared to individual-based interventions, population-based interventions were few and mostly targeted reduction in sodium intake and tobacco control strategies. These were very cost-effective with many being cost-saving.

Conclusions

This evidence synthesis provides a contemporary update on interventions that offer good value for money in LMICs. Population-based interventions especially those targeting reduction in salt intake and tobacco control are very cost-effective in LMICs with potential to generate economic gains that can be reinvested to improve health and/or other sectors. While this evidence is relevant for policy across these regions, decision makers should additionally take into account other multi-sectoral perspectives, including considerations in budget impact, fairness, affordability and implementation while setting priorities for resource allocation.

Electronic supplementary material

The online version of this article (10.1186/s12962-018-0108-9) contains supplementary material, which is available to authorized users.

Cost-Effectiveness Thresholds: the Past, the Present and the Future

Cost-effectiveness (CE) thresholds are being discussed more frequently and there have been many new developments in this area; however, there is a lack of understanding about what thresholds mean and their implications. This paper provides an overview of the CE threshold literature. First, the meaning of a CE threshold and the key assumptions involved (perfect divisibility, marginal increments in budget, etc.) are highlighted using a hypothetical example, and the use of historic/heuristic estimates of the threshold is noted along with their limitations. Recent endeavours to estimate the empirical value of the thresholds, both from the supply side and the demand side, are then presented. The impact on CE thresholds of future directions for the field, such as thresholds across sectors and the incorporation of multiple criteria beyond quality-adjusted life-years as a measure of 'value', are highlighted. Finally, a number of common issues and misconceptions associated with CE thresholds are addressed.

A cost-utility analysis of small bite sutures versus large bite sutures in the closure of midline laparotomies in the United Kingdom National Health Service

Purpose

This study aimed to perform an economic evaluation of small bite sutures versus large bite sutures in the closure of midline laparotomies in the United Kingdom National Health Service (NHS).

Methods

A cost-utility analysis was conducted using data from a systematic literature review. Large bite sutures placed 10 mm from the wound edge were compared to small bite sutures 3–6 mm from the wound edge. The analysis used a 3-year time horizon in order to take into account complications including incisional hernias and surgical site infections (SSIs). Cost and benefit data were considered from the perspective of the NHS. A two-way sensitivity analysis was conducted to assess the impact of a variation in the clinical effectiveness of small bite sutures.

Results

The incremental cost-effectiveness ratio was calculated to be −£482.61 per quality-adjusted life year (QALY) using the proposed small bite suture technique, indicating a cost saving to the NHS. Sensitivity analysis demonstrated that small bites are a cost-neutral technique provided that the cost of using small bites is less than £98 per patient. Small bites cost less than £20,000/QALY when they reduce either the rate of SSIs by more than 15% or the rate of hernias by more than 3.4%.

Conclusion

This study proposes that small bite sutures should become the mainstay suturing technique in the closure of midline laparotomies, replacing large bite sutures, which dominate current practice. The financial savings accompanied by the decrease in SSI rates and herniation warrant the use of this new technique. The sensitivity analysis demonstrates that findings hold true for a wide range of levels of clinical effectiveness for small bites.

Cost-effectiveness thresholds: methods for setting and examples from around the world

INTRODUCTION

Cost-effectiveness thresholds (CETs) are used to judge if an intervention represents sufficient value for money to merit adoption in healthcare systems. The study was motivated by the Brazilian context of HTA, where meetings are being conducted to decide on the definition of a threshold.

AREAS COVERED

An electronic search was conducted on Medline (via PubMed), Lilacs (via BVS) and ScienceDirect followed by a complementary search of references of included studies, Google Scholar and conference abstracts. Cost-effectiveness thresholds are usually calculated through three different approaches: the willingness-to-pay, representative of welfare economics; the precedent method, based on the value of an already funded technology; and the opportunity cost method, which links the threshold to the volume of health displaced. An explicit threshold has never been formally adopted in most places. Some countries have defined thresholds, with some flexibility to consider other factors. An implicit threshold could be determined by research of funded cases.

EXPERT COMMENTARY

CETs have had an important role as a 'bridging concept' between the world of academic research and the 'real world' of healthcare prioritization. The definition of a cost-effectiveness threshold is paramount for the construction of a transparent and efficient Health Technology Assessment system.

Economic evaluations of alcohol prevention interventions: Is the evidence sufficient? A review of methodological challenges

•

There are few economic evaluations of alcohol prevention interventions.

•

Consideration of impacts beyond an individual’s health in evaluations is limited.

•

No published studies using other priority-setting methods in the alcohol area.

•

Consideration of wider societal perspectives and health inequalities is minimal.

•

Including inter-sectoral costs and consequences in evaluations is challenging.

Public health interventions have unique characteristics compared to health technologies, which present additional challenges for economic evaluation (EE). High quality EEs that are able to address the particular methodological challenges are important for public health decision-makers. In England, they are even more pertinent given the transition of public health responsibilities in 2013 from the National Health Service to local government authorities where new agents are shaping policy decisions.

Addressing alcohol misuse is a globally prioritised public health issue. This article provides a systematic review of EE and priority-setting studies for interventions to prevent and reduce alcohol misuse published internationally over the past decade (2006–2016). This review appraises the EE and priority-setting evidence to establish whether it is sufficient to meet the informational needs of public health decision-makers.

619 studies were identified via database searches. 7 additional studies were identified via hand searching journals, grey literature and reference lists. 27 met inclusion criteria. Methods identified included cost-utility analysis (18), cost-effectiveness analysis (6), cost-benefit analysis (CBA) (1), cost-consequence analysis (CCA) (1) and return-on-investment (1). The review identified a lack of consideration of methodological challenges associated with evaluating public health interventions and limited use of methods such as CBA and CCA which have been recommended as potentially useful for EE in public health. No studies using other specific priority-setting tools were identified.