Health technology assessment in Canada. A decade in review

La evaluación de tecnologías sanitarias en el Ministerio de Salud Pública de Ecuador como herramienta para la compra de medicamentos entre 2012 y 2015

Objetivo.

Conocer el uso de la evaluación de tecnologías sanitarias (ETS) en la toma de decisiones del Ministerio de Salud Pública (MSP) del Ecuador para la compra de medicamentos que no se encuentran en el Cuadro Nacional de Medicamentos Básicos (CNMB).

Métodos.

Con la información de las bases de datos de la Dirección de Inteligencia de la Salud (DIS) y la Dirección Nacional de Medicamentos y Dispositivos Médicos (DNMDM), se compararon las decisiones tomadas por ambas instancias, para conocer el uso y la congruencia de los informes de ETS en las decisiones de compra de los medicamentos no incluidos en el CNMB.

Resultados.

Entre 2012 y 2015, se han elaborado 227 informes, de los cuales 87 corresponden a medicamentos, 36 a dispositivos, 29 a procedimientos médicos, 34 a programas sanitarios, y 41 a otras tecnologías médicas. De los informes de medicamentos, 59 fueron solicitados por la DNM. La concordancia entre las decisiones tomadas por las dos direcciones que participan en el proceso alcanzó 80%.

Conclusiones.

La ETS se inició en el MSP en 2012 a través de la DIS. Considerando que la mayoría de informes evalúan medicamentos, es indispensable que se desarrollen informes para otros tipos de tecnologías médicas y que se difunda al máximo su desarrollo y uso. A pesar de que el nivel de concordancia entre las decisiones es elevado, es importante seguir mejorando el alcance y la calidad de los informes, así como monitorizar la incorporación y difusión de las tecnologías autorizadas y financiadas para conocer la efectividad y el impacto de la ETS en Ecuador.

What can economics add to health technology assessment? Please not just another cost-effectiveness analysis!

Evidence based medicine is not only important for clinical practice, but national governments have embraced it through health technology assessment (HTA). HTA combines data from randomized controlled trials (RCT) and observational studies with an economic component (among other issues). HTA, however, is not taking full advantage of economics. This paper presents five areas in which economics may improve not only HTA, but the RCT methods that underpin it. HTA needs to live up to its original agenda of being a interdisciplinary field and draw methods not just from biostatistics, but from a range of discipline, including economics. By focusing only on cost effectiveness analysis (CEA), however, we go nowhere close to fulfilling this potential.

Using health technology assessment to support evidence-based decision-making in Canada: an academic perspective

Health technology assessment (HTA) adopts a multidisciplinary approach to comprehensively assess safety, efficacy, effectiveness, economic and organizational impact, and any potential social and ethical implications associated with adoption and diffusion of a technology in a healthcare system. Canada was one of the first pioneers in using HTA as a research tool to support evidence-based decision-making. This article describes the current application of HTA in Canada, with a focus on some federal and Ontario initiatives in which the authors have extensive knowledge and experience to illustrate how academic researchers conduct HTA in collaboration with decision makers. Some issues and challenges are also highlighted that will hopefully stimulate a broader discussion among HTA stakeholders to move HTA forward.

Supporting the use of health technology assessments in policy making about health systems

Objectives:The objective of this study is to profile the health technology assessments (HTAs) produced in Canada and other selected countries and assess their potential to inform policy making about health systems in jurisdictions other than the ones for which they were produced, and to develop and pilot test prototypes for packaging and assessing the relevance of HTAs for health system managers and policy makers.

Methods:We compiled an inventory of all HTAs that were produced by nine HTA agencies between September 2003 and August 2006; coded the title and abstract of each HTA according to the technologies assessed, methods used, and whether or not context-specific actionable messages were provided; developed a prototype for a structured, decision-relevant HTA summary and for a relevance-assessment form; and pilot-tested the prototypes using semistructured telephone interviews with a purposive sample of Canadian healthcare managers and policy makers.

Results:Our review of the 223 HTAs identified that: (i) 44 HTAs addressed health system arrangements (20 percent); (ii) 205 incorporated a systematic review (92 percent), whereas only 12 incorporated a sociopolitical assessment using explicit methods (5 percent); and (iii) 50 contained context-specific actionable messages (22 percent). Our interviews identified significant support for both the general idea of an HTA summary and the prototype's specific elements, but mixed views about using peer assessments of relevance.

Conclusions:Those involved in supporting the use of HTAs in policy making about health systems may wish to produce structured decision-relevant summaries for their systematic review-containing HTAs to increase the prospects for their HTAs being used outside the jurisdiction for which they were produced.

Ethics in Canadian health technology assessment: A descriptive review

Background: Despite the mandate to examine the medical, ethical, and economic implications of the development and use of health technology, health technology assessment (HTA) reports often emphasize the epidemiologic and economic aspects, and omit ethical considerations. This study examines both whether and how ethical issues are incorporated into HTA.

Objectives: We aim to (i) review a set of Canadian HTA reports for ethics content, (ii) describe the strategies used to incorporate ethically relevant information into HTA, and (iii) determine the presence of implicit ethical issues in a sample of HTA reports.

Methods: Descriptive and qualitative content analysis of 608 HTA reports produced by six Canadian HTA agencies from January 1997 to December 2006.

Results: We found that (i) a minority (17 percent) of Canadian HTA reports addressed ethical issues, (ii) secondary research predominates while primary analysis is rare, (iii) implicit ethical issues are present in HTA reports that do not purport to address ethics.

Conclusions: Canadian HTA reports rarely explicitly, and then only superficially, address ethics, though implicit ethical issues abound.

Evidence-informed health policy 2 - survey of organizations that support the use of research evidence

Background

Previous surveys of organizations that support the development of evidence-informed health policies have focused on organizations that produce clinical practice guidelines (CPGs) or undertake health technology assessments (HTAs). Only rarely have surveys focused at least in part on units that directly support the use of research evidence in developing health policy on an international, national, and state or provincial level (i.e., government support units, or GSUs) that are in some way successful or innovative or that support the use of research evidence in low- and middle-income countries (LMICs).

Methods

We drew on many people and organizations around the world, including our project reference group, to generate a list of organizations to survey. We modified a questionnaire that had been developed originally by the Appraisal of Guidelines, Research and Evaluation in Europe (AGREE) collaboration and adapted one version of the questionnaire for organizations producing CPGs and HTAs, and another for GSUs. We sent the questionnaire by email to 176 organizations and followed up periodically with non-responders by email and telephone.

Results

We received completed questionnaires from 152 (86%) organizations. More than one-half of the organizations (and particularly HTA agencies) reported that examples from other countries were helpful in establishing their organization. A higher proportion of GSUs than CPG- or HTA-producing organizations involved target users in the selection of topics or the services undertaken. Most organizations have few (five or fewer) full-time equivalent (FTE) staff. More than four-fifths of organizations reported providing panels with or using systematic reviews. GSUs tended to use a wide variety of explicit valuation processes for the research evidence, but none with the frequency that organizations producing CPGs, HTAs, or both prioritized evidence by its quality. Between one-half and two-thirds of organizations do not collect data systematically about uptake, and roughly the same proportions do not systematically evaluate their usefulness or impact in other ways.

Conclusion

The findings from our survey, the most broadly based of its kind, both extend or clarify the applicability of the messages arising from previous surveys and related documentary analyses, such as how the 'principles of evidence-based medicine dominate current guideline programs' and the importance of collaborating with other organizations. The survey also provides a description of the history, structure, processes, outputs, and perceived strengths and weaknesses of existing organizations from which those establishing or leading similar organizations can draw.

Economic evaluations conducted by Canadian health technology assessment agencies: Where do we stand?

Objectives:To examine the production of Health Technology Assessments (HTAs) with economic evaluations (EEs) conducted by Canadian HTA agencies.

Methods:This research used a three-step approach: (i) the Web sites of five Canadian organizations promoting HTA were searched to identify HTA reports with EEs; (ii) HTA agencies were surveyed to verify that our information was complete with respect to HTA activities and to describe the factors that influence the HTA process in Canada (i.e., selection of HTA topics, execution, dissemination of results and future trends); (iii) HTAs with EEs were appraised in terms of study design, retrieval of clinical and economic evidence, resource utilization and costing, effectiveness measures, treatment of uncertainty as well as presence of a budget impact analysis (BIA), and policy recommendations.

Results:Two hundred forty-nine HTA reports were identified of which 19 percent included EEs (n= 48). Decision analytic techniques were used in approximately 75 percent of the forty-eight EEs and probabilistic sensitivity analyses were commonly used by all agencies to deal with parameter uncertainty. BIAs or policy recommendations were given in 50 percent of the evaluations. Differences between agencies were observed in terms of selection of topics, focus of assessment and production of HTA (e.g., in-house activities). Major barriers to the conduct of HTAs with EEs were capacity, a lack of interest by decision makers and a lack of robust clinical information.

Conclusions:The results of this research point to the need for increased HTA training, collaboration, evidence synthesis, and use of pragmatic “real world” evaluations.

Harmonization of evidence requirements for health technology assessment in reimbursement decision making

As more countries use HTA to inform decisions on the reimbursement of health technologies, harmonization of evidence requirements between jurisdictions has been proposed, mainly on the grounds of improved efficiency. Harmonization has the potential to avoid duplication of effort for both manufacturers and HTA bodies involved in preparing and reviewing HTA submissions for innovative technologies. However, it also carries risks of loss of local control over decisions, the application of general data standards which are not universally accepted and slowing the rate of development of innovation in the analytical disciplines supporting HTA. This study reviews the issues associated with harmonization taking into account the perspectives of the multiple stakeholders. This study draws on experiences from recent initiatives intended to promote the harmonization of HTA and experience from related fields, particularly regulatory approval of new medical technologies.

Health technology assessment in Canada: diversity and evolution

Canada has health technology assessment programs at national, provincial and local levels. The programs have been complementary in providing advice to decision makers in health care. A national strategy for the management of health technologies is expected to strengthen communication with policy areas.

An evaluation of methods used in health technology assessments produced for the Medical Services Advisory Committee

OBJECTIVE

To examine the methods used in health technology assessments (HTAs) produced for the Medical Services Advisory Committee (MSAC) reviewing the effectiveness of a technology or procedure.

DESIGN AND SETTING

Data were extracted from the effectiveness section of HTA application assessment reports published between 1 January 1998 and 17 July 2006 and available on the MSAC website. Only HTAs of effectiveness interventions were examined, as the methods used to undertake such reviews are well established.

MAIN OUTCOME MEASURES

Variables reflecting methods used in the HTAs to evaluate the effectiveness of health technologies or procedures.

RESULTS

Of 56 MSAC HTA reports available, 31 met the inclusion criteria. Considerable variability was shown to exist between the various indicators of quality and the methodology used within the HTAs. Reports did not describe potential conflicts of interest of participants. The majority of reports (19/31) did not formally state the research question that the assessment was attempting to answer. Just over half of the reports (18/31) provided details of validity assessment of the included studies.

CONCLUSIONS

Minimum and consistent standards of methodology and reporting are required in Australian HTAs, using international recommendations of best practice to increase the transparency and applicability of these reports.

Lean systems approaches to health technology assessment: a patient-focused alternative to cost-effectiveness analysis

Many countries now use health technology assessment (HTA) to review new and emerging technologies, especially with regard to reimbursement, pricing and/or clinical guidelines. One of the common, but not universal, features of these systems is the use of economic evaluation, normally cost-effectiveness analysis (CEA), to confirm that new technologies offer value for money. Many have criticised these systems as primarily being concerned with cost containment, rather than advancing the interests of patients or innovators. This paper calls into question the underlying principles of CEA by arguing that value in the healthcare system may in fact be unconstrained. It is suggested that 'lean management principles' can be used not only to trim waste from the health system, but as a method of creating real incentives for innovation and value creation. Following the lean paradigm, this value must be defined purely from the patients' perspective, and the entire health system needs to work towards the creation of such value. This paper offers as a practical example a lean approach to HTA, arguing that such an approach would lead to better incentives for innovation in health, as well as more patient-friendly outcomes in the long run.

Recommendations in health technology assessments worldwide

Objectives:The objective of this study is to analyze statistically the possible determinants and implications of including or not including recommendations in health technology assessments (HTAs).

Methods:A sample of 433 HTAs published by eleven leading institutions or agencies in nine countries was reviewed and analyzed statistically by multiple logistic regression.

Results:The extent of policy and research recommendations in HTAs varies greatly from country to country. The content and scope of HTAs have some impact on recommendations. Extensive assessment of economic and organizational aspects increases the likelihood of including policy recommendations. Extensive assessment of technological and patient aspects increases the likelihood of including research recommendations, whereas extensive focus on economic aspects is negatively related to research recommendations. The most striking result is that the use of external partners for assessment increases the likelihood of including research recommendations in HTAs but not policy recommendations.

Conclusions:HTA commissioners, agencies, institutions, and funding authorities need to be aware of the consequences of the choices they make in advance of assessing health technologies. Outsourcing HTA to external partners suggests a greater likelihood of being told that “more research is needed.” The scope and content of HTAs has an impact on the type of recommendations, and country-specific preferences are strong predictors of recommendations in HTAs.

What influences the choice of assessment methods in health technology assessments? Statistical analysis of international health technology assessments from 1989 to 2002

Objectives:Health technology assessment (HTA) has been used as input in decision making worldwide for more than 25 years. However, no uniform definition of HTA or agreement on assessment methods exists, leaving open the question of what influences the choice of assessment methods in HTAs. The objective of this study is to analyze statistically a possible relationship between methods of assessment used in practical HTAs, type of assessed technology, type of assessors, and year of publication.

Methods:A sample of 433 HTAs published by eleven leading institutions or agencies in nine countries was reviewed and analyzed by multiple logistic regression.

Results:The study shows that outsourcing of HTA reports to external partners is associated with a higher likelihood of using assessment methods, such as meta-analysis, surveys, economic evaluations, and randomized controlled trials; and with a lower likelihood of using assessment methods, such as literature reviews and “other methods.” The year of publication was statistically related to the inclusion of economic evaluations and shows a decreasing likelihood during the year span. The type of assessed technology was related to economic evaluations with a decreasing likelihood, to surveys, and to “other methods” with a decreasing likelihood when pharmaceuticals were the assessed type of technology.

Conclusions:During the period from 1989 to 2002, no major developments in assessment methods used in practical HTAs were shown statistically in a sample of 433 HTAs worldwide. Outsourcing to external assessors has a statistically significant influence on choice of assessment methods.

Time-trends in health technology assessments: An analysis of developments in composition of international health technology assessments from 1989 to 2002

Objectives: Health Technology Assessment (HTA) as a method for producing evidence in the health-care sector has been used for more than 25 years but has grown in extent during the past years. The objective of this study is to explore a possible evolution in these HTAs, in type of assessed technologies, in type of assessors, and in its methods.

Methods: A structured literature review was conducted of 433 HTA reports from the period 1989 to 2002 by eleven leading HTA institutions worldwide. The review focused on the methodology used, the assessors, and the assessed technologies and was designed to elucidate general time-trends in the practical application of HTA.

Results: The study shows that literature reviews are still the most often used method of assessment and accounts for a relatively stable fraction of assessments. The fraction of economic evaluations shows a slightly decreasing trend in contrast to randomized controlled trials and modeling/evaluations, which are applied more frequently. The data also demonstrate a more frequent use of external partners as assessors and a shift between devices and pharmaceuticals as assessed technologies.

Conclusions: The study shows an increase in the number of HTAs but no major developments in assessment methods used and, therefore, no widespread spillover from the development in research methods in general to the field of HTA methodology.

Use of health technology assessment in decision making: Coresponsibility of users and producers?

Objectives: Health technology assessment (HTA) is a policy-oriented form of research designed to inform decision-makers on the introduction, use, and dissemination of health technology. Whereas research on knowledge transfer has focused on knowledge producers, little attention has been given to the user's perspective. This study examines how health-care provider, administrator, and patient associations across Canada use HTA reports and the limitations they encounter when accessing and using scientific knowledge.

Methods: This study draws from semistructured interviews (n = 42) conducted with three types of user, located in British Columbia, Alberta, Saskatchewan, Ontario, and Quebec. Applying well-established conceptual categories in knowledge utilization research, our qualitative analyses sought to define more precisely how HTA is used by interviewees as well as the most significant barriers they encounter.

Results: The vast majority of users recognize the usefulness and credibility of HTA reports. Of interest, the way they use HTA takes different forms. Although administrators and health-care providers are in a better position than patient associations to act directly on HTA messages—making an instrumental use of HTA—we also found conceptual and symbolic uses across all groups. Our results also indicate that significant organizational, scientific, and material limitations hinder the use of scientific evidence. Overcoming such barriers requires a greater commitment from both HTA producers and users.

Conclusions: This study argues that, to ensure better uptake of HTA, it should become a shared responsibility between HTA producers and various types of user.

International comparison of the definition and the practical application of health technology assessment

OBJECTIVES

Health Technology Assessment (HTA) is defined as a policy research approach that examines the short- and long-term social consequences of the application or use of technology. Internationally different institutions have translated this definition to local contexts. In Denmark, HTA is comprehensive with focus on four aspects of the problem in question (technology [clinical evidence], economy, patient, and organization). The objective of this study is to study how the application of HTA differs across leading countries and to study the extent to which Danish HTA reports differ from foreign HTAs.

METHODS

A sample of 433 HTA reports published in the period 1989--2002 by eleven leading institutions or agencies in Denmark and eight other countries were reviewed. We looked at the characteristics of the HTA with respect to focus on the four main aspects and the manner in which each aspect has been approached.

RESULTS

The study shows health technology procedures to be the most common type of health technology assessed in HTAs and literature review to be the most often used method of analysis. Policy recommendations are only present in approximately half of the HTA reports.

CONCLUSIONS

In the HTAs one generally sees a great focus on the clinical aspect of health technologies, leaving the economic, the patient-related, and the organizational aspect much more unanalyzed. The Danish HTAs generally have a wider scope than HTAs produced in other countries and tend to focus more frequently on patient-related and organizational dimensions.

Redefining health technology assessment in Canada: diversification of products and contextualization of findings

OBJECTIVES

While strategies for enhancing the dissemination and impact of Health Technology Assessment (HTA) are now being increasingly examined, the characteristics of HTA production have received less attention.

METHODS

This study presents the results of a content analysis of the HTA documents (n = 187) produced by six Canadian agencies from 1995 to 2001, supplemented by interviews with chief executive officers and researchers (n = 40). The goal of this analysis was to characterize the agencies' portfolios and to analyze the challenges these agencies face in responding to the increased demand for HTA.

RESULTS

On average, thirty HTA products were issued annually by the agencies. While the bulk of documents produced were full HTA reports (76 percent), two agencies showed significant diversification in their products. Three agencies in particular actively supported the publication of results in scientific journals. Three agencies showed evidence of adapting to different institutional environments by specializing in certain areas (drugs, health services). Overall, a significant portion of the agencies' HTAs contained data on costs (37 percent) and effectiveness (48 percent), whereas ethical and social issues were rarely addressed (17 percent). Most agencies addressed issues and outcomes that did not strictly fall under the typical definition of HTA but that increased the "contextualization" of their findings.

CONCLUSIONS

Our discussion highlights four paradoxes and reflects further on challenges raised by the coordination of HTA within large countries and among European states. This study concludes that HTA is being redefined in Canada as HTA agencies offer a more contextualized informational basis, an approach that may prove more compatible with the increased demand for HTA.

Devolving healthcare delivery to regional health authorities: is health technology assessment prepared to follow?

Since the establishment of health technology assessment units in the latter 1980s, Canada has witnessed an unprecedented transformation of the governance, management and service delivery of its healthcare system. In Alberta, this transformation culminated in the establishment of regional health authorities that provide integrated healthcare to Albertans. With the shift of responsibility for healthcare delivery from the provincial to the regional level, the Alberta Heritage Foundation for Medical Research HTA unit recognized that for health technology assessment to continue to be relevant, it must follow this change. Four steps were taken to refocus the unit's scope: a thorough analysis of the healthcare environment; face-to-face interviews with the chief executive officers of the regions; the development of a framework for HTA in the regions; and the organization of a conference on evidence-based decision making. These steps were helpful in bringing HTA to the attention of regional decision makers. A formal, analytical assessment of the regional healthcare environment, provision of general information (through the framework and conference) and individual information (through face-to-face interviews) enabled a proactive engagement with regions. However, to meet the demands and needs of a population that expects comprehensive coverage that delivers "state of the art" diagnostics and treatments, the efficacy and effectiveness of interventions can sometimes be of subordinate importance.

Co-ordinating health technology assessment in Canada: a European perspective

Health Technology Assessment (HTA) is well-established in Canada, with both a national co-ordinating office for HTA and several provincial HTA organisations. The success of and relationship between these agencies, may provide useful lessons for policy-makers considering how to facilitate increased co-operation and harmonisation of HTA activity across Europe. Interviews were conducted with a number of key individuals associated with HTA in Canada to identify achievements, examine the relationship between the provincial agencies and the national co-ordinating office, and detect barriers to the increased use of HTA knowledge in decision-making.

Trends in surgery for gastroesophageal reflux disease: the effect of laparoscopic surgery on utilization

BACKGROUND

In addition to substituting for open surgery, minimally invasive surgery may lower thresholds for intervention and thus increase overall utilization rates. The degree to which laparoscopy may have lowered the threshold for elective anti-reflux surgery is unknown.

METHODS

Using the Uniform Hospital Discharge Dataset and ICD-9 procedure and diagnosis codes, we identified all laparoscopic and open anti-reflux procedures performed on adults in Massachusetts, New Hampshire, and Vermont for each year from 1993 to 1998. We then examined secular trends and regional variation in the use of laparoscopic and open anti-reflux surgery.

RESULTS

The population-based rate of anti-reflux surgery more than doubled between 1993 (4.8 per 100,000) and 1998 (11.7 per 100,000). Laparoscopic anti-reflux procedures increased more than 6-fold between 1993 and 1998, from 1.2 to 8.9 procedures per 100,000 adults, with accompanying declines in overall length of stay and mortality. However, the number of open anti-reflux procedures decreased only modestly (22%) over this time period. In the year hospitals performed their first laparoscopic anti-reflux operation, procedure rates nearly tripled, on average, and then increased slowly in subsequent years. In 1997 and 1998, rates of anti-reflux surgery varied nearly 5-fold across hospital referral regions, ranging from 5.4 to 24.5 per 100,000.

CONCLUSIONS

With the growth of minimally invasive surgery, rates of anti-reflux surgery have increased substantially, with wide regional variation in intervention rates. Further research is needed to determine the appropriate threshold for surgical treatment in patients with gastroesophageal reflux.

Influence of evidence-based guidance on health policy and clinical practice in England

OBJECTIVES

To examine the influence of evidence-based guidance on health care decisions, a study of the use of seven different sources and types of evidence-based guidance was carried out in senior health professionals in England with responsibilities either for directing and purchasing health care based in the health authorities, or providing clinical care to patients in trust hospitals or in primary care.

DESIGN

Postal survey.

SETTING

Three health settings: 46 health authorities, 162 acute and/or community trust hospitals, and 96 primary care groups in England.

SAMPLE

566 subjects (46 directors of public health, 49 directors of purchasing, 375 clinical directors/consultants in hospitals, and 96 lead general practitioners).

MAIN OUTCOME MEASURES

Knowledge of selected evidence-based guidance, previous use ever, beliefs in quality, usefulness, and perceived influence on practice.

RESULTS

A usable response rate of 73% (407/560) was achieved; 82% (334/407) of respondents had consulted at least one source of evidence-based guidance ever in the past. Professionals in the health authorities were much more likely to be aware of the evidence-based guidance and had consulted more sources (mean number of different guidelines consulted 4.3) than either the hospital consultants (mean 1.9) or GPs in primary care (mean 1.8). There was little variation in the belief that the evidence-based guidance was of "good quality", but respondents from the health authorities (87%) were significantly more likely than either hospital consultants (52%) or GPs (57%) to perceive that any of the specified evidence-based guidance had influenced a change of practice. Across all settings, the least used route to accessing evidence-based guidance was the Internet. For several sources an effect was observed between use ever, the health region where the health professional worked, and the region where the guidance was produced or published. This was evident for some national sources as well as in those initiatives produced locally with predominantly local distribution networks.

CONCLUSIONS

The evidence-based guidance specified was significantly more likely to be seen to have contributed to the decisions of public health specialists and commissioners than those of consultants in hospitals or of GPs in a primary care setting. Appropriate information support and dissemination systems that increase awareness, access, and use of evidence-based guidance at the clinical interface should be developed.