HTA AND INNOVATION OF VALUE: GETTING TO KNOW YOU

Fostering innovation for societal gain has been a core attribute of technology assessment (TA) since its inception. Although many observers were concerned that TA would be a means by which government would impede the development and use of technology, this was not the intent of policy makers or the agencies that conducted the original TAs.

Broader economic evaluation of disease management programs using multi-criteria decision analysis

OBJECTIVES

The aim of this paper is to develop a methodological framework to facilitate the application of Multi-Criteria Decision Analysis (MCDA) for a comprehensive economic evaluation of disease management programs (DMPs).

METHODS

We studied previously developed frameworks for the evaluation of DMPs and different methods of MCDA and we used practical field experience in the economic evaluation of DMPs and personal discussions with stakeholders in chronic care.

RESULTS

The framework includes different objectives and criteria that are relevant for the evaluation of DMPs, indicators that can be used to measure how DMPs perform on these criteria, and distinguishes between the development and implementation phase of DMPs. The objectives of DMPs are categorised into a) changes in the process of care delivery, b) changes in patient lifestyle and self-management behaviour, c) changes in biomedical, physiological and clinical health outcomes, d) changes in health-related quality of life, and e) changes in final health outcomes. All relevant costs of DMPs are also included in the framework. Based on this framework we conducted a MCDA of a hypothetical DMP versus usual care.

CONCLUSIONS

We call for a comprehensive economic evaluation of DMPs that is not just based on a single criterion but takes into account multiple relevant criteria simultaneously. The framework we presented here is a step towards standardising such an evaluation.

From efficacy to equity: Literature review of decision criteria for resource allocation and healthcare decisionmaking

Objectives

Resource allocation is a challenging issue faced by health policy decisionmakers requiring careful consideration of many factors. Objectives of this study were to identify decision criteria and their frequency reported in the literature on healthcare decisionmaking.

Method

An extensive literature search was performed in Medline and EMBASE to identify articles reporting healthcare decision criteria. Studies conducted with decisionmakers (e.g., focus groups, surveys, interviews), conceptual and review articles and articles describing multicriteria tools were included. Criteria were extracted, organized using a classification system derived from the EVIDEM framework and applying multicriteria decision analysis (MCDA) principles, and the frequency of their occurrence was measured.

Results

Out of 3146 records identified, 2790 were excluded. Out of 356 articles assessed for eligibility, 40 studies included. Criteria were identified from studies performed in several regions of the world involving decisionmakers at micro, meso and macro levels of decision and from studies reporting on multicriteria tools. Large variations in terminology used to define criteria were observed and 360 different terms were identified. These were assigned to 58 criteria which were classified in 9 different categories including: health outcomes; types of benefit; disease impact; therapeutic context; economic impact; quality of evidence; implementation complexity; priority, fairness and ethics; and overall context. The most frequently mentioned criteria were: equity/fairness (32 times), efficacy/effectiveness (29), stakeholder interests and pressures (28), cost-effectiveness (23), strength of evidence (20), safety (19), mission and mandate of health system (19), organizational requirements and capacity (17), patient-reported outcomes (17) and need (16).

Conclusion

This study highlights the importance of considering both normative and feasibility criteria for fair allocation of resources and optimized decisionmaking for coverage and use of healthcare interventions. This analysis provides a foundation to develop a questionnaire for an international survey of decisionmakers on criteria and their relative importance. The ultimate objective is to develop sound multicriteria approaches to enlighten healthcare decisionmaking and priority-setting.

Field testing of a multicriteria decision analysis (MCDA) framework for coverage of a screening test for cervical cancer in South Africa

BACKGROUND

Systematic and transparent approaches to priority setting are needed, particularly in low-resource settings, to produce decisions that are sound and acceptable to stakeholders. The EVIDEM framework brings together Health Technology Assessment (HTA) and multi-criteria decision analysis (MCDA) by proposing a comprehensive set of decision criteria together with standardized processes to support decisionmaking. The objective of the study was to field test the framework for decisionmaking on a screening test by a private health plan in South Africa.

METHODS

Liquid-based cytology (LBC) for cervical cancer screening was selected by the health plan for this field test. An HTA report structured by decision criterion (14 criteria organized in the MCDA matrix and 4 contextual criteria) was produced based on a literature review and input from the health plan. During workshop sessions, committee members 1) weighted each MCDA decision criterion to express their individual perspectives, and 2) to appraise LBC, assigned scores to each MCDA criterion on the basis of the by-criterion HTA report.Committee members then considered the potential impacts of four contextual criteria on the use of LBC in the context of their health plan. Feedback on the framework and process was collected through discussion and from a questionnaire.

RESULTS

For 9 of the MCDA matrix decision criteria, 89% or more of committee members thought they should always be considered in decisionmaking. Greatest weights were given to the criteria "Budget impact", "Cost-effectiveness" and "Completeness and consistency of reporting evidence". When appraising LBC for cervical cancer screening, the committee assigned the highest scores to "Relevance and validity of evidence" and "Disease severity". Combination of weights and scores yielded a mean MCDA value estimate of 46% (SD 7%) of the potential maximum value. Overall, the committee felt the framework brought greater clarity to the decisionmaking process and was easily adaptable to different types of health interventions.

CONCLUSIONS

The EVIDEM framework was easily adapted to evaluating a screening technology in South Africa, thereby broadening its applicability in healthcare decision making.

Bridging health technology assessment (HTA) with multicriteria decision analyses (MCDA): field testing of the EVIDEM framework for coverage decisions by a public payer in Canada

BACKGROUND

Consistent healthcare decision making requires systematic consideration of decision criteria and evidence available to inform them. This can be tackled by combining multicriteria decision analysis (MCDA) and Health Technology Assessment (HTA). The objective of this study was to field-test a decision support framework (EVIDEM), explore its utility to a drug advisory committee and test its reliability over time.

METHODS

Tramadol for chronic non-cancer pain was selected by the health plan as a case study relevant to their context. Based on extensive literature review, a by-criterion HTA report was developed to provide synthesized evidence for each criterion of the framework (14 criteria for the MCDA Core Model and 6 qualitative criteria for the Contextual Tool). During workshop sessions, committee members tested the framework in three steps by assigning: 1) weights to each criterion of the MCDA Core Model representing individual perspective; 2) scores for tramadol for each criterion of the MCDA Core Model using synthesized data; and 3) qualitative impacts of criteria of the Contextual Tool on the appraisal. Utility and reliability of the approach were explored through discussion, survey and test-retest. Agreement between test and retest data was analyzed by calculating intra-rater correlation coefficients (ICCs) for weights, scores and MCDA value estimates.

RESULTS

The framework was found useful by the drug advisory committee in supporting systematic consideration of a broad range of criteria to promote a consistent approach to appraising healthcare interventions. Directly integrated in the framework as a "by-criterion" HTA report, synthesized evidence for each criterion facilitated its consideration, although this was sometimes limited by lack of relevant data. Test-retest analysis showed fair to good consistency of weights, scores and MCDA value estimates at the individual level (ICC ranging from 0.676 to 0.698), thus lending some support for the reliability of the approach. Overall, committee members endorsed the inclusion of most framework criteria and revealed important areas of discussion, clarification and adaptation of the framework to the needs of the committee.

CONCLUSIONS

By promoting systematic consideration of all decision criteria and the underlying evidence, the framework allows a consistent approach to appraising healthcare interventions. Further testing and validation are needed to advance MCDA approaches in healthcare decisionmaking.