International lessons in new methods for grading and integrating cost effectiveness evidence into clinical practice guidelines

Cost-effectiveness of intraoperative radiation therapy versus intensity-modulated radiation therapy for the treatment of early breast cancer: a disinvestment analysis

BACKGROUND

Adjuvant radiotherapy represents a key component in curative-intent treatment for early-stage breast cancer patients. In recent years, two accelerated partial breast irradiation (APBI) techniques are preferred for this population in our organization: electron-based Intraoperative radiation therapy (IORT) and Linac-based External Beam Radiotherapy, particularly Intensity-modulated radiation therapy (IMRT). Recently published long-term follow-up data evaluating these technologies have motivated a health technology reassessment of IORT compared to IMRT.

METHODS

We developed a Markov model to simulate health-state transitions from a cohort of women with early-stage breast cancer, after lumpectomy and adjuvant APBI using either IORT or IMRT techniques. The cost-effectiveness from a private health provider perspective was assessed from a disinvestment point of view, using life-years (LYs) and recurrence-free life-years (RFLYs) as measure of benefits, along with their respective quality adjustments. Expected costs and benefits, and the incremental cost-effectiveness ratio (ICER) were reported. Finally, a sensitivity and scenario analyses were performed to evaluate the cost-effectiveness using lower IORT local recurrence and metastasis rates in IORT patients, and if equipment maintenance costs are removed.

RESULTS

IORT technology was dominated by IMRT in all cases (i.e., fewer benefits with greater costs). Despite small differences were found regarding benefits, especially for LYs, costs were considerably higher for IORT. For sensitivity analyses with lower recurrence and metastasis rates for IORT, and scenario analyses without equipment maintenance costs, IORT was still dominated by IMRT.

CONCLUSIONS

For this cohort of patients, IMRT was, at least, non-inferior to IORT in terms of expected benefits, with considerably lower costs. As a result, IORT disinvestment should be considered, favoring the use of IMRT in these patients.

The economics of the COVID-19 pandemic: economic evaluation of government mitigation and suppression policies, health system innovations, and models of care

Background

The COVID-19 pandemic has impacted the scope of health economics literature, which will increasingly examine value beyond health care interventions such as government policy and broad health system innovations.

Aim

The study analyzes economic evaluations and methodologies evaluating government policies suppressing or mitigating transmission and reducing COVID-19, broad health system innovations, and models of care. This can facilitate future economic evaluations and assist government and public health policy decisions during pandemics.

Methods

The Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) was used. Methodological quality was quantified using the scoring criteria in European Journal of Health Economics, Consolidated Health Economic Evaluation Reporting Standards (CHEERS) 2022 Checklist and the National Institute for Health and Care Excellence's (NICE) Cost Benefit Analysis Checklist. PUBMED, Medline, and Google Scholar were searched from 2020-2021.

Results

Cost utility analysis (CUA) and cost benefit analysis (CBA) analyzing mortality, morbidity, quality adjusted life year (QALY) gained, national income loss, and value of production effectively evaluate government policies suppressing or mitigating COVID-19 transmission, disease, and impacting national income loss. The WHO's pandemic economic framework facilitates economic evaluations of social and movement restrictions. Social return on investment (SROI) links benefits to health and broader social improvements. Multi-criteria decision analysis (MCDA) can facilitate vaccine prioritization, equitable health access, and technology evaluation. Social welfare function (SWF) can account for social inequalities and population-wide policy impact. It is a generalization of CBA, and operationally, it is equal to an equity-weighted CBA. It can provide governments with a guideline for achieving the optimal distribution of income, which is vital during pandemics. Economic evaluations of broad health system innovations and care models addressing COVID-19 effectively use cost effectiveness analysis (CEA) that utilize decision trees and Monte Carlo models, and CUAs that effectively utilize decision trees and Markov models, respectively.

Conclusion

These methodologies are very instructive for governments, in addition to their current use of CBA and the value of a statistical life analytical tool. CUA and CBA effectively evaluate government policies suppressing or mitigating COVID-19 transmission, disease, and impacts on national income loss. CEA and CUA effectively evaluate broad health system innovations and care models addressing COVID-19. The WHO's framework, SROI, MCDA, and SWF can also facilitate government decision-making during pandemics.

Supplementary Information

The online version contains supplementary material available at 10.1007/s10389-023-01919-z.

Are intersectoral costs considered in economic evaluations of interventions relating to sexually transmitted infections (STIs)? A systematic review

BACKGROUND/OBJECTIVE

Sexually transmitted infections (STIs) not only have an impact on the health sector but also the private resources of those affected, their families and other sectors of society (i.e. labour, education). This study aimed to i) review and identify economic evaluations of interventions relating to STIs, which aimed to include a societal perspective; ii) analyse the intersectoral costs (i.e. costs broader than healthcare) included; iii) categorise these costs by sector; and iv) assess the impact of intersectoral costs on the overall study results.

METHODS

Seven databases were searched: MEDLINE (PubMed), EMBASE (Ovid), Web of Science, CINAHL, PsycINFO, EconLit and NHS EED. Key search terms included terms for economic evaluation, STIs and specific infections. This review considered trial- and model-based economic evaluations conducted in an OECD member country. Studies were included that assessed intersectoral costs. Intersectoral costs were extracted and categorised by sector using Drummond's cost classification scheme (i.e. patient/family, productivity, costs in other sectors). A narrative synthesis was performed.

RESULTS

Twenty-nine studies qualified for data extraction and narrative synthesis. Twenty-eight studies applied a societal perspective of which 8 additionally adopted a healthcare or payer perspective, or both. One study used a modified payer perspective. The following sectors were identified: patient/family, informal care, paid labour (productivity), non-paid opportunity costs, education, and consumption. Patient/family costs were captured in 11 studies and included patient time, travel expenses, out-of-pocket costs and premature burial costs. Informal caregiver support (non-family) and unpaid help by family/friends was captured in three studies. Paid labour losses were assessed in all but three studies. Three studies also captured the costs and inability to perform non-paid work. Educational costs and future non-health consumption costs were each captured in one study. The inclusion of intersectoral costs resulted in more favourable cost estimates.

CONCLUSIONS

This systematic review suggests that economic evaluations of interventions relating to STIs that adopt a societal perspective tend to be limited in scope. There is an urgent need for economic evaluations to be more comprehensive in order to allow policy/decision-makers to make better-informed decisions.

Health economic evaluation of digital nursing technologies: a review of methodological recommendations

BACKGROUND

Health economic evaluation of digital nursing technologies (DNT) is important to provide information that helps avoid undesirable developments and implementations as well as increase the chances of success of developed applications. At the same time, studies and evidence on cost-effectiveness are still very rare in this field. Review studies in related technology areas such as telemedicine frequently criticise the quality and comparability of health economic evaluations conducted in this field. Based on a content analysis of methodological literature on the economic evaluation of innovative (digital) technologies in health and nursing, this article aims to identify specific challenges in this research area and offers recommendations on how to address these challenges to promote more sound health economic evaluations in the future.

METHODS

A rapid review was conducted, consisting of a systematic search in the Pubmed database as well as Google Scholar. In addition, the literature lists of the analysed texts were scoured for additional texts to be included. Methodological literature, single studies, and reviews were included. A total of 536 studies were screened, of which 29 were included in the full text analysis.

RESULTS

Based on the systematic content analysis of the studies under consideration, 10 specific methodological challenges are identified, and the methodological recommendations were examined for consideration. A particular focus was given to whether specific methodological approaches might be needed in the context of evaluating the efficiency of DNT.

CONCLUSION

Many of the challenges identified for the health economic evaluations of digital nursing technologies are comparable to those of other complex health care interventions. The recommendations discussed can help to alleviate those challenges. Future research should focus on alternative approaches to assessing causality in different phases of technology development while maintaining high evidence standards. High-evidence economic assessment of technologies in nursing care should be carried out in routine use, especially if they are intended to be reimbursed by the social insurance.

Patient and Payer Preferences for Additional Value Criteria

Background: Defining the value of healthcare is an elusive target, and depends heavily on the decision context and stakeholders involved. Cost-utility analysis and the quality-adjusted life year (QALY) have become the method and value definition of choice for traditional value judgements in coverage and pricing decisions. Other criteria that may influence value are often not measured and therefore omitted from value assessments, or are only used to qualitatively contextualize assessments. The objective of this study was to engage two key stakeholders; patients and payers to elicit and rank the importance of additional value criteria, potentially assessed in Multiple Criteria Decision Analysis (MCDA).

Methods: This study consisted of a focus group with cancer patients (n = 7), including follow-up questions through an electronic survey, and in-depth phone interviews with payers (n = 5).

Results: For payers, value equated either with criteria that provided tangible benefits (from their perspective) such as new treatment options that respond to serious unmet need. For patients, population-level value equated to options that would potentially benefit them in the future and the value of hope. However, these criteria were seen by payers as difficult to measure and incorporate into objective decision making.

Limitations: The findings from this study are primarily limited due to generalizability. Due to the small sample size, it was outside the scope of this study to calculate a weight for each criterion that could be used as part of a quantitative MCDA.

Conclusion: MCDA, with particular attention to qualitative aspects, is an avenue to incorporate these additional criteria into value assessments, as well as provide an opportunity for reflecting the patient’s preferences in assessing the value of a treatment.

Potential Criteria for Frameworks to Support the Evaluation of Innovative Medicines in Upper Middle-Income Countries-A Systematic Literature Review on Value Frameworks and Multi-Criteria Decision Analyses

Background

Multicriteria Decision Analysis (MCDA), a formal decision support framework, has been growing in popularity recently in the field of health care. MCDA can support pricing and reimbursement decisions on the macro level, which is of great importance especially in countries with more limited resources.

Objectives

The aim of this systematic review was to facilitate the development of future MCDA frameworks, by proposing a set of criteria focusing on the purchasing decisions of single-source innovative pharmaceuticals in upper middle-income countries.

Methods

A systematic literature review was conducted on the decision criteria included in value frameworks (VFs) or MCDA tools. Scopus, Medline, databases of universities, websites of Health Technology Assessment Agencies, and other relevant organizations were included in the search. Double title-abstract screening and double full-text review were conducted, and all extracted data were double-checked. A team of researchers performed the merging and selection process of the extracted criteria.

Results

A total of 1,878 articles entered the title and abstract screening. From these, 341 were eligible to the full-text review, and 36 were included in the final data extraction phase. From these articles 394 criteria were extracted in total. After deduplication and clustering, 26 different criteria were identified. After the merging and selection process, a set of 16 general criteria was proposed.

Conclusion

Based on the results of the systematic literature review, a pool of 16 criteria was selected. This can serve as a starting point for constructing MCDA frameworks in upper middle-income countries after careful adaptation to the local context.

Stakeholders' engagement in co-producing policy-relevant knowledge to facilitate employment for persons with developmental disabilities

BACKGROUND

Persons with developmental disabilities (PWDD) face a number of individual, environmental and societal barriers when seeking employment. Integrated knowledge translation (IKT) involves ongoing and dynamic interactions between researchers and stakeholders for the purpose of engaging in mutually beneficial research to address these types of multi-faceted barriers. There is a knowledge gap in the IKT literature on effective stakeholder engagement strategies outside of the dissemination stage to inform policy. In this paper, we report on a number of engagement strategies employed over a 2-year period to engage a wide range of stakeholders in different stages of an IKT project that aimed to investigate the 'wicked' problem of employment for PWDD.

METHOD

Our engagement plan included multiple linked strategies and was designed to ensure the meaningful engagement of, and knowledge co-production with, stakeholders. We held two participatory consensus-building stakeholder policy dialogue events to co-produce knowledge utilising the nominal group technique and the modified Delphi technique. A total of 31 and 49 stakeholders engaged in the first and second events, respectively, from six key stakeholder groups. Focused engagement strategies were employed to build on the stakeholder dialogues for knowledge mobilisation and included a focus group attended only by PWDD, a stakeholder workshop attended only by policy/decision-makers, a webinar attended by human resources professionals and employers, and a current affairs panel attended by the general public.

RESULTS

Our findings suggest that the level of engagement for each stakeholder group varies depending on the goal and need of the project. Our stakeholder dialogue findings highlight the inherent challenges in co-framing and knowledge co-production through the meaningful engagement of multiple stakeholders who hold different ideas and interests. Focused outreach is needed to foster relationships and trust for meaningful engagement.

CONCLUSIONS

In addition to providing guidance on how to implement adaptable meaningful engagement strategies, these findings contribute to discussions on how IKT projects are planned and funded. More studies to explore effective mechanisms for engaging a wide range of stakeholders in IKT research are needed. More evidence of successful engagement strategies employed by researchers to achieve meaningful knowledge co-production is also key to advancing the discipline.

Multi-criteria decision analysis for health technology assessment: addressing methodological challenges to improve the state of the art

BACKGROUND

Multi-criteria decision analysis (MCDA) concepts, models and tools have been used increasingly in health technology assessment (HTA), with several studies pointing out practical and theoretical issues related to its use. This study provides a critical review of published studies on MCDA in the context of HTA by assessing their methodological quality and summarising methodological challenges.

METHODS

A systematic review was conducted to identify studies discussing, developing or reviewing the use of MCDA in HTA using aggregation approaches. Studies were classified according to publication time and type, country of study, technology type and study type. The PROACTIVE-S approach was constructed and used to analyse methodological quality. Challenges and limitations reported in eligible studies were collected and summarised; this was followed by a critical discussion on research requirements to address the identified challenges.

RESULTS

129 journal articles were eligible for review, 56% of which were published in 2015-2017; 42% focused on pharmaceuticals; 36, 26 and 18% reported model applications, issues regarding MCDA implementation analyses, and proposing frameworks, respectively. Poor compliance with good methodological practice (< 25% complying studies) was found regarding behavioural analyses, discussion of model assumptions and uncertainties, modelling of value functions, and dealing with judgment inconsistencies. The five most reported challenges related to evidence and data synthesis; value system differences and participant selection issues; participant difficulties; methodological complexity and resource balance; and criteria and attributes modelling. A critical discussion on ways to address these challenges ensues.

DISCUSSION

Results highlight the need for advancement in robust methodologies, procedures and tools to improve methodological quality of MCDA in HTA studies. Research pathways include developing new model features, good practice guidelines, technologies to enable participation and behavioural research.

Prioritizing barriers and solutions to improve employment for persons with developmental disabilities

Purpose: Persons with a developmental disability have the lowest rate of labour force participation relative to other disabilities. The widening gap between the labour force participation of persons with versus without disability has been an enduring concern for many governments across the globe, which has led to policy initiatives such as labour market activation programs, welfare reforms, and equality laws. Despite these policies, persistently poor labour force participation rates for persons with developmental disabilities suggest that this population experiences pervasive barriers to participating in the labour force.Materials and methods: In this study, a two-phase qualitative research design was used to systematically identify, explore and prioritize barriers to employment for persons with developmental disabilities, potential policy solutions and criteria for evaluating future policy initiatives. Incorporating diverse stakeholder perspectives, a Nominal Group Technique and a modified Delphi technique were used to collect and analyze data.Results: Findings indicate that barriers to employment for persons with developmental disabilities are multi-factorial and policy solutions to address these barriers require stakeholder engagement and collaboration from multiple sectors.Conclusions: Individual, environmental and societal factors all impact employment outcomes for persons with developmental disabilities. Policy and decision makers need to address barriers to employment for persons with developmental disabilities more holistically by designing policies considering employers and the workplace, persons with developmental disabilities and the broader society. Findings call for cross-sectoral collaboration using a Whole of Government approach.Implications for RehabilitationPersons with a developmental disability face lower levels of labour force participation than any other disability group.Individual, environmental and societal factors all impact employment outcomes for persons with developmental disabilities.Decision and policy makers need to address barriers to employment for persons with developmental disabilities holistically through policies guiding employers and broader societal behaviour in addition to those aimed at the individuals (such as skill development or training).Due to multi-factorial nature of barriers to employment for persons with developmental disabilities, policy solutions are wide-ranging and fall under the responsibility of multiple sectors for implementation. This calls for cross-sectoral collaboration using a "Whole of Government" approach, with shared goals and integrated responses.

Comparison of weighting methods used in multicriteria decision analysis frameworks in healthcare with focus on low- and middle-income countries

AIM

Criteria weighting is a key element of multicriteria decision analysis that is becoming extensively used in healthcare decision-making. In our narrative review we describe the advantages and disadvantages of various weighting methods.

METHODS

An assessment of the eight identified primary criteria weighting methods was compiled on domains including their resource requirements, and potential for bias.

RESULTS

In general, we found more complex methods to have less potential for bias; however, resource intensity and general participant burden is greater for these methods.

CONCLUSION

The selection of the most appropriate method depends on the decision-making context. The simple multiattribute rating technique (SMART) method combined with swing-weighting technique and the analytic hierarchy process methods may be the most feasible approaches for low- and middle-income countries.

Access criteria for anti-TNF agents in spondyloarthritis: influence on comparative 1-year cost-effectiveness estimates

Background

Anti-tumor necrosis factor (anti-TNF) agents are an effective, but costly, treatment for spondyloarthritis (SpA). Worldwide, multiple sets of access criteria aim to restrict anti-TNF therapy to patients with specific clinical characteristics, yet the influence of access criteria on anti-TNF cost-effectiveness is unknown. Our objective was to use data from the DESIR cohort, a prospective study of early SpA patients in France, to determine whether the French anti-TNF access criteria are the most cost-effective in that setting relative to other potential restrictions.

Methods

We used data from the DESIR cohort to create five study populations of patients meeting anti-TNF access criteria from Canada, France, Germany, United Kingdom, and Hong Kong, respectively. For each study population, we calculated the costs and quality-adjusted life years (QALYs) over 1 year of patients treated and not treated with anti-TNF therapy. To control for differences between anti-TNF users and non-users, we used linear regression models to derive adjusted mean costs and QALYs. We calculated incremental cost-effectiveness ratios (ICERs) representing the incremental cost per additional QALY gained by treating with an anti-TNF within each of the five study populations, using bootstrapping to explore the range of uncertainty in costs and QALYs. A series of sensitivity analyses was conducted, including one to simulate the effect of a 24-week stopping rule for anti-TNF non-responders.

Results

Anti-TNF access criteria from France were satisfied by the largest proportion of DESIR patients (27.8%), followed by Germany (25.1%), Canada (23.8%), the UK (12.1%) and Hong Kong (8.6%). Confidence intervals around incremental costs and QALYs in the basecase analysis were overlapping, indicating that anti-TNF cost-effectiveness estimates derived from each subset were similar. In the sensitivity analysis that examined the effect of excluding costs accumulated past 24 weeks by anti-TNF non-responders, the incremental cost per QALY was reduced by approximately 25% relative to the basecase analysis (France: €857,992 vs. €1,105,859; Canada: € 626,459 vs. €818,186; Germany: € 422,568 vs. €545,808); UK €578,899 vs. €766,217; Hong Kong €335,418 vs. €456,850).

Conclusions

Anti-TNF cost-effectiveness is strongly affected by treatment continuation among non-responders. Access criteria could improve anti-TNF cost-effectiveness by defining patients likely to respond.

Electronic supplementary material

The online version of this article (doi:10.1186/s12962-017-0081-8) contains supplementary material, which is available to authorized users.