A Value Framework for the Assessment of Diagnostic Technologies: A Proposal Based on a Targeted Systematic Review and a Multistakeholder Deliberative Process in Latin America

Documenting the Full Value of Vaccination: A Systematic Review of Value Frameworks

OBJECTIVES

Economic evaluations of vaccination may not fully account for nonhealth patient impacts on families, communities, and society (ie, broader value elements). Omission of broader value elements may reflect a lack of established measurement methodology, lack of agreement over which value elements to include in economic evaluations, and a lack of consensus on whether the value elements included should vary by vaccination type or condition. We conducted a systematic review of value frameworks to identify broader value elements and measurement guidance that may be useful for capturing the full value of vaccination.

METHODS

We searched Ovid MEDLINE, PubMed, Embase, and the gray literature to identify value frameworks for all health interventions, and we extracted information on each framework's context, value elements, and any available guidance on how these elements should be measured. We used descriptive statistics to analyze and compare the prevalence of broader value elements in vaccination value frameworks and other healthcare-related value frameworks.

RESULTS

Our search identified 62 value frameworks that met inclusion criteria, 9 of which were vaccination specific. Although vaccination frameworks included several broader value elements, such as reduced transmissibility and public health benefits, the elements were represented inconsistently across the frameworks. Vaccination frameworks omitted several value elements included in nonvaccination-specific frameworks, including dosing and administration complexity and affordability. In addition, guidance for measuring broader value elements was underdeveloped.

CONCLUSIONS

Future efforts should further evaluate inclusion of broader value elements in economic evaluations of vaccination and develop standards for their subsequent measurement.

How is Value Defined in Molecular Testing in Cancer? A Scoping Review

OBJECTIVE

To identify how value is defined in studies that focus on the value of molecular testing in cancer and the extent to which broadening the conceptualisation of value in healthcare has been applied in the molecular testing literature.

METHODS

A scoping review was undertaken using Joanna Briggs Institute (JBI) guidance. Medline, Embase, EconLit and Cochrane Library were searched in August 2023. Articles were eligible if they reported costs relative to outcomes, novel costs, or novel outcomes of molecular testing in cancer. Results were synthesised and qualitative content analysis was performed with deductive and inductive frameworks.

RESULTS

Ninety-one articles were included in the review. The majority (75/91) were conventional economic analyses (comparative economic evaluations and budget impact assessments) and undertaken from a healthcare system perspective (38/91). Clinical outcomes dominate the assessment of value (61/91), with quality-adjusted life-years (QALYs) the most common outcome measure (45/91). Other definitions of value were diverse (e.g. psychological impact, access to trials), inconsistent, and largely not in keeping with evolving guidance.

CONCLUSIONS

Broader concepts of value were not commonly described in the molecular testing literature focusing on cancer. Conventional approaches to measuring the health costs and outcomes of molecular testing in cancer prevail with little focus on non-clinical elements of value. There are emerging reports of non-clinical outcomes of testing information, particularly psychological consequences. Intrinsic attributes of the testing process and preferences of those who receive testing information may determine the realised societal value of molecular testing and highlight challenges to implementing such a value framework.

Opportunities to Enhance Diagnostic Testing and Antimicrobial Stewardship: A Qualitative Multinational Survey of Healthcare Professionals

INTRODUCTION

Antimicrobial resistance (AMR) is a global public health challenge. Global efforts to decrease AMR through antimicrobial stewardship (AMS) initiatives include education and optimising the use of diagnostic technologies and antibiotics. Despite this, economic and societal challenges hinder AMS efforts. The objective of this study was to obtain insights from healthcare professionals (HCPs) on current challenges and identify opportunities for optimising diagnostic test utilisation and AMS efforts.

METHODS

Three hundred HCPs from six countries (representing varied gross national incomes per capita, healthcare system structure, and AMR rates) were surveyed between November 2022 through January 2023. A targeted literature review and expert interviews were conducted to inform survey development. Descriptive statistics were used to summarise survey responses.

RESULTS

These findings suggest that the greatest challenges to diagnostic test utilisation were economic in nature; many HCPs reported that AMS initiatives were lacking investment (32.3%) and resourcing (40.3%). High resistance rates were considered the greatest barriers to appropriate antimicrobial use (52.0%). Most HCPs found local and national guidelines to be very useful (≥ 51.0%), but areas for improvement were noted. The importance of AMS initiatives was confirmed; diagnostic practices were acknowledged to have a positive impact on decreasing AMR (70.3%) and improving patient outcomes (81.0%).

CONCLUSION

AMS initiatives, including diagnostic technology utilisation, are pivotal to decreasing AMR rates. Interpretation of these survey results suggests that while HCPs consider diagnostic practices to be important in AMS efforts, several barriers to successful implementation still exist including patient/institutional costs, turnaround time of test results, resourcing, AMR burden, and education. While some barriers differ by country, these survey results highlight areas of opportunities in all countries for improved use of diagnostic technologies and broader AMS efforts, as perceived by HCPs. Greater investment, resourcing, education, and updated guidelines offer opportunities to further strengthen global AMS efforts.

A Systematic Review of Value Criteria for Next-Generation Sequencing/Comprehensive Genomic Profiling to Inform Value Framework Development

OBJECTIVES

To comprehensively identify and map an exhaustive list of value criteria for the assessment of next-generation sequencing/comprehensive genomic profiling (NGS/CGP), to be used as an aid in decision making.

METHODS

We conducted a systematic review to identify existing value frameworks (VFs) applicable to any type of healthcare technology. VFs and criteria were mapped to a previously published Latin American (LA) VF to harmonize definitions and identify additional criteria and or subcriteria. Based on this analysis, we extracted a comprehensive, evidence-based list of criteria and subcriteria to be considered in the design of a NGS/CGP VF.

RESULTS

A total of 42 additional VFs were compared with the LA VF, 88% were developed in high-income countries, 30% targeted genomic testing, and 16% specifically targeted oncology. A total of 242 criteria and subcriteria were extracted; 227 (94%) were fully/partially included in the LA VF; and 15 (6%) were new. Clinical benefit and economic aspects were the most common criteria. VFs oriented to genomic testing showed significant overlap with other VFs. Considering all criteria and subcriteria, a total of 18 criteria and 36 individual subcriteria were identified.

CONCLUSIONS

Our study provides an evidence-based set of criteria and subcriteria for healthcare decision making useful for NGS/CGP as well as other health technologies. The resulting list can be beneficial to inform decision making and will serve as a foundation to co-create a multistakeholder NGS/CGP VF that is aligned with the needs and values of health systems and could help to improve patient access to high-value technologies.

Toward the Implementation of a Value Framework for Diagnostic Technologies: Operationalization, Piloting, and Validation in Latin America

OBJECTIVES

In 2020, a group of 30 stakeholders from Latin America established 15 criteria for a diagnostic technologies value framework (D-VF) to help assess and inform decisions on diagnostic technologies. This article aims to present the operationalization, piloting, and initial validation of the framework for its implementation.

METHODS

This work was carried out collaboratively with a variety of stakeholders. Three sequential phases were undertaken: (1) operationalization of the D-VF through a literature search for conceptual definitions and assessment tools, (2) piloting of the D-VF through a rapid health technology assessment document applying the methodology of the framework, and (3) a face validation process conducted through a virtual workshop, where usefulness and implementation aspects of the framework were assessed.

RESULTS

The operationalization of the framework was conducted, and a methodological user guide was published. The D-VF criteria were applied in a health technology assessment document on human papilloma virus testing in cervical cancer screening. Also, an open-access training program was developed. Stakeholders agreed on the usefulness of the D-VF for assessment and decision-making stages of diagnostic technologies. However, they highlighted the need to improve technical capacities and the potential for added complexity when applying a D-VF with many criteria. The absence of an established value framework for diagnostic technologies in Latin America and the potential for strengthening technical capacities made the project valuable to those involved.

CONCLUSIONS

The diagnostic technologies value framework was shown to be fit for implementation in real-life decision-making settings after the operationalization, piloting, and initial validation phases. Further experiences are important to support its implementation.

A Value Framework to Assess Patient-Facing Digital Health Technologies That Aim to Improve Chronic Disease Management: A Delphi Approach

OBJECTIVES

Digital health technologies (DHTs) can optimise healthcare costs and improve quality and efficiency of care. However, the fast-paced rate of innovation and varying evidence standards can make it difficult for decision-makers to assess these technologies in an efficient and evidence-based manner. We sought to develop a comprehensive framework to assess the value of novel patient-facing DHTs used to manage chronic diseases by eliciting stakeholder value preferences.

METHODS

Literature review and primary data collection from a three-round web-Delphi exercise was utilized. 79 participants from 5 stakeholder groups (patients, physicians, industry, decision makers, and influencers) and 3 countries (United States of America, United Kingdom, and Germany) took part. Likert scale data were statistically analyzed to determine intergroup differences in both country and stakeholder groups, stability of results, and overall consensus.

RESULTS

The resulting co-created framework comprised 33 stable indicators with consensus from quantitative value judgments across domains: health inequalities, data rights and governance, technical and security, economic characteristics, clinical characteristics, and user preferences. Lack of stakeholder consensus was observed on the importance of value-based care models, optimizing resources for sustainable systems, and stakeholder involvement in DHT design, development, and implementation; however, this was because of high rates of neutrality and not negative judgments. Supply-side actors and academic experts were the most unstable stakeholder groups.

CONCLUSION

Stakeholder value judgments revealed a need for a coordinated regulatory and health technology assessment policy response that updates laws to meet technological innovations, offers a pragmatic approach to evidence standards to assess DHTs, and involves stakeholders to understand and meet their needs.

The Value of New: Consideration of Product Novelty in Health Technology Assessments of Pharmaceuticals

BACKGROUND

Efforts to understand how treatments affect patients and society have broadened the criteria that health technology assessment (HTA) organizations apply to value assessments. We examined whether HTA agencies in eight countries consider treatment novelty in methods and deliberations.

METHODS

We defined a novel pharmaceutical product to be one that offers a new approach to treatment (e.g., new mechanism of action), addresses an unmet need (e.g., targets a rare condition without effective treatments), or has a broader impact beyond what is typically measured in an HTA. We reviewed peer-reviewed publications and technical guidance materials from HTA organizations in Australia, Canada, England, France, The Netherlands, Norway, Sweden, and the United States (US). In addition, we explored how HTA organizations integrated novelty considerations into deliberations and recommendations related to two newer therapies-voretigene neparvovec for an inherited retinal disorder and ocrelizumab for multiple sclerosis.

RESULTS

None of the HTA organizations acknowledge treatment novelty as an explicit value criterion in their assessments of pharmaceutical products. However, drugs that have novel characteristics are given special consideration, particularly when they address an unmet need. Several organizations document a willingness to expend more resources and accept greater evidence uncertainty for such treatments. Qualitative deliberations about the additional unquantified potential benefits of treatment may also influence HTA recommendations.

CONCLUSION

Major HTA organizations do not recognize novelty as an explicit value criterion, although drugs with novel characteristics may receive special consideration. There is an opportunity for organizations to codify their approach to evaluating novelty in value assessment.

Measuring clinical outcomes of highly multiplex molecular diagnostics for respiratory infections: A systematic review and conceptual framework

Objectives

To review methodologies and outcomes reporting among these studies and to develop a conceptual framework of outcomes to assist in guiding studies and production of clinical metrics.

Data sources

PubMed and Embase from January 1, 2012, thru December 1, 2021.

Study eligibility criteria

Studies evaluating highly multiplex molecular respiratory diagnostics and their impact on either clinical or economic outcomes.

Methods

A systematic literature review (SLR) of methodologies and outcomes reporting was performed. A qualitative synthesis of identified SLRs and associated primary studies was conducted to develop conceptual framework for outcomes.

Results

Ultimately, 4 systemic literature reviews and their 12 associated primary studies were selected for review. Most primary studies included patient outcomes focusing on antimicrobial exposure changes such as antibiotic (80%) and antiviral use (50%) or occupancy changes such as hospital length of stay (60%). Economic outcomes were infrequently reported, and societal outcomes, such as antibiotic resistance impact, were absent from the reviewed literature. Qualitative evidence synthesis of reported outcomes yielded a conceptual framework of outcomes to include operational, patient, economic, and societal domains.

Conclusions

Our review highlights the significant heterogeneity in outcomes reporting among clinical impact studies for highly multiplex molecular respiratory diagnostics. Furthermore, we developed a conceptual framework of outcomes domains that may act as a guide to improve considerations in outcomes selection and reporting when evaluating clinical impact of these tests. These improvements may be important in synthesizing the evidence for informing clinical decision making, guidelines, and financial reimbursement.

Development, testing, and implementation of a new procedure to assess the clinical added benefit of pharmaceuticals

OBJECTIVES

The reimbursement process for innovative health technologies in Hungary lacks any formalized assessment of clinical added benefit (CAB). The aim of this research is to present the development, retrospective testing, and implementation of a local assessment framework for determining the CAB of cancer treatments at the Department of Health Technology Assessment of the National Institute of Pharmacy and Nutrition in Hungary.

METHODS

The assessment framework was drafted after screening existing methods and a retrospective comparison of local reimbursement dossiers to that of German and French methods. The Magnitude of Clinical Benefit Scale of the European Society for Medical Oncology was chosen to rate the extent of CAB in oncology, as part of a conclusion complemented by the assessment of endpoint relevance and the quality of evidence. Several rounds of retrospective assessments have been conducted involving all clinical assessors, iterated with semistructured discussions to consolidate divergence between assessors. External stakeholders were consulted to provide feedback on the framework.

RESULTS

Retrospective assessments resulted in average more than 75 percent concordance between assessors on each element of the conclusion. Input from ten stakeholders was also incorporated; stakeholders were generally supportive, and they mostly commented on the concept, the elements of the framework, and its implementation.

CONCLUSIONS

The procedure is suitable for routine use in the decision-making process to describe the CAB of antineoplastic technologies in Hungary. Further extension of the framework is required to cover more disease areas for structured and comparable conclusions on CAB of innovative health technologies.

Optimizing the delivery of genetic and advanced diagnostic testing in the province of Ontario: challenges and implications for laboratory technology assessment and management in decentralized healthcare systems

AIMS

The Canadian province of Ontario provides full coverage for its residents (pop.14.8 M) for hospital-based diagnostic testing. Historical governance of the healthcare system and a legacy scheme of health technology assessment (HTA) and financing has led to a suboptimal approach of adopting advanced diagnostic technology (i.e. protein expression, cytogenetic, and molecular/genetic) for guiding therapeutic decisions. The aim of this research is to explore systemic barriers and provide guidance to improve patient and care provider experiences by reducing delays and inequity of access to testing, while benefitting laboratory innovators and maximizing system efficiency.

MATERIALS AND METHODS

A mixed-methods approach including literature review, semi-structured interviews, and a multi-stakeholder forum involving patient representatives (n = 1), laboratory leaders (n = 6), physicians (n = 5), Ministry personnel (n = 4), administrators (n = 3), extra-provincial experts, and researchers (n = 7), as well as pharmaceutical (n = 5) and diagnostic companies (n = 2). The forum considered evidence of good practices in adoption, implementation, and financing laboratory services and identified barriers as well as feasible options for improving advanced diagnostic testing in Ontario.

RESULTS

Overarching challenges identified included: barriers to define what is needed; need for a clear approach to adoption; and the need for more oversight and coordination. Recommendations to address these included a shift to an anticipatory system of test adoption, creating a fit-for-purpose system of health technology management that consolidates existing evaluation processes, and modernizing the governance and financing of testing so that it is managed at a care-delivery level.

CONCLUSIONS

The proposals for change in Ontario highlight the role that HTA, governance, and financing of health technology play along the continuum of a health technology life cycle within a healthcare system where decision-making is highly decentralized. Resource availability and capacity were not a concern - instead, solutions require higher levels of coordination and system integration along with innovative approaches to HTA.