Time-trends in health technology assessments: An analysis of developments in composition of international health technology assessments from 1989 to 2002

HEALTH TECHNOLOGY ASSESSMENT: THE SCIENTIFIC CAREER OF A POLICY CONCEPT

Objectives: The aim of this work was to provide a comprehensive overview of the evolution of the health technology assessment (HTA) concept in the scientific literature through a scientometric approach.

Methods: A literature search was conducted, by selecting publications, as well as news from the media, containing “health technology assessment” in their title, abstracts, or keywords. We then undertook a bibliometric and network analysis on the corpus of 2,865 publications thus obtained.

Results: Since a first publication in 1978, interest in HTA remained marginal until a turning point in the late 1980s, when growth of the number of publications took off alongside the creation of the U.K.’s NICE agency. Since then, publications have spread across several journals. The ranking of the organizations that publish such articles does not reflect any hegemonic position. However, HTA-related scientific production is strongly concentrated in Commonwealth and Nordic countries. Despite its transnational aspects, research on HTA has been framed within a small number of scientific networks and by a few opinion leaders.

Conclusions: The “career” of the HTA concept may be seen as a scientific-knowledge based institutionalization of a public policy. To succeed in a country, HTA first needs scientific prerequisites, such as an organized scientific community working on the health sector and health services. Then, it appears that the recognition of this research by decision makers plays a key role in the development of the field.

The application of qualitative approaches to the evaluation of telehealth systems in Australia

Health Technology Assessment (HTA) of telehealth applications is necessary to assist policy making and reimbursement strategies. A number of authors have concluded that because of the complexities of health, both qualitative and quantitative data are required to resolve questions about the appropriate use of diffuse technologies such as telehealth. Both telepsychiatry and teleradiology are already covered for reimbursement in Australia. However, the majority of telehealth applications are not suitable for assessment using traditional HTA quantitative methodology. Assessment of telehealth applications should be based more on qualitative methods involving surveys of the experience of practitioners and patients, rather than relying solely on traditional quantitative methods.

The Role of decision-analytic modelling in German health technology assessments

BACKGROUND

Decision-analytic modelling (DAM) has become a widespread method in health technology assessments (HTA), but the extent to which modelling is used differs among international HTA institutions. In Germany, the use of DAM is optional within HTAs of the German Institute of Medical Documentation and Information (DIMDI). Our study examines the use of DAM in DIMDI HTA reports and its effect on the quality of information provided for health policies.

METHODS

A review of all DIMDI HTA reports (from 1998 to September 2012) incorporating an economic assessment was performed. All included reports were divided into two groups: HTAs with DAM and HTAs without DAM. In both groups, reports were categorized according to the quality of information provided for healthcare decision making.

RESULTS

Of the sample of 107 DIMDI HTA reports, 17 (15.9%) used DAM for economic assessment. In the group without DAM, conclusions were limited by the quality of economic information in 51.1% of the reports, whereas we did not find limited conclusions in the group with DAM. Furthermore, 24 reports without DAM (26.7%) stated that using DAM would likely improve the quality of information of the economic assessment.

CONCLUSION

The use of DAM techniques can improve the quality of HTAs in Germany. When, after a systematic review of existing literature within a HTA, it is clear that DAM is likely to positively affect the quality of the economic assessment DAM should be used.

Exploring qualitative research synthesis: the role of patients' perspectives in health policy design and decision making

Health systems are placing more and more emphasis on the design and delivery of services that are focused on the patient, and there is a growing interest in patient involvement in health policy research and health technology assessment (HTA). Furthermore, there is a growing research interest in eliciting patients' views, not only on 'what works' for patients but also on the need for intervention and on factors influencing the implementation of particular health technologies, their appropriateness and acceptability. This article focuses on qualitative research synthesis in eliciting patients' perspectives. Its aim is to bring research closer to policy development and decision making, to facilitate better use of research findings for health and welfare, to generate a body of evidence, and to ensure that effective and appropriate information is used in health policy decision design. A variety of synthesizing approaches in qualitative research are explored, such as meta-synthesis, meta-summary, meta-ethnography, and meta-study, focusing especially on methodology. Meta-synthesis and meta-ethnography are probably the most frequently cited approaches in qualitative research synthesis and have perhaps the most developed methodology. The implications of these various synthesizing approaches in relation to health policy and HTA are discussed, and we suggest that meta-synthesis and meta-summary are particularly useful approaches. They have an explicit focus on 'evidence synthesis', fairly clear methodologies, and they are designed to not only present interpretations of the findings but also integrate research findings. Qualitative research synthesis enables researchers to synthesize findings from multiple qualitative studies on patients' perspectives instead of establishing new, expensive, and perhaps redundant studies that might intrude on the lives of patients. Qualitative research synthesis is highly recommended by decision makers and in health policy research and HTA. In cases where patient assessment is important to overall success, it can provide those responsible for policy and decision making with a broad and varied range of knowledge about patients' perspectives before they make decisions on the application of health technologies.

Health technology assessment: Research trends and future priorities in Europe

Objective:

To provide an overview of health services research related to health technology assessment (HTA) and to identify research priorities from a European perspective.

Methods:

Several methods were used: systematic review of articles indexed with the MeSH term ‘technology assessment’ in PubMed from February 1999-2009; online survey among experts; and conference workshop discussions.

Results:

Research activity in HTA varies considerably across Europe. The research was categorised into six areas: (1) the breadth of analysis in HTA (such as economic, organizational and social aspects); (2) HTA products developed to meet the needs of policy-makers (such as horizon scanning, mini-HTA, and core HTA); (3) handling life-cycle perspectives in relation to technologies; (4) topics that challenge existing methods and for which HTA should be developed to address the themes more comprehensively (such as public health interventions and organizational interventions); (5) development of HTA capacity and programmes; and (6) links between policy and HTA. An online survey showed that the three areas that were given priority were the relationship between HTA and policy-making (71%), the impact of HTA (62%) and incorporating patient aspects in HTA (50%). Policy-makers highlighted HTA and innovation processes as their main research priority (42%). Areas that the systematic review identified as future priorities include issues within the six existing research areas such as disinvestment, developing evidence for new technologies, assessing the wider effects of technology use, and determining how HTA affects decision-making. In addition, relative effectiveness and individualized treatments are areas of growing interest.

Conclusions:

The research priorities identified are important for obtaining high quality and cost-effective health care in Europe. Managing the introduction, use and phasing out of technologies challenges health services throughout Europe, and these processes need to be improved to successfully manage future more general challenges. An ageing population and a diminishing workforce both require strong efforts to ensure effective and well-organized use of human resources and technologies. Furthermore, Europe needs to focus on innovation. This is closely linked to use of technologies and calls for future research.

Supporting the use of health technology assessments in policy making about health systems

Objectives:The objective of this study is to profile the health technology assessments (HTAs) produced in Canada and other selected countries and assess their potential to inform policy making about health systems in jurisdictions other than the ones for which they were produced, and to develop and pilot test prototypes for packaging and assessing the relevance of HTAs for health system managers and policy makers.

Methods:We compiled an inventory of all HTAs that were produced by nine HTA agencies between September 2003 and August 2006; coded the title and abstract of each HTA according to the technologies assessed, methods used, and whether or not context-specific actionable messages were provided; developed a prototype for a structured, decision-relevant HTA summary and for a relevance-assessment form; and pilot-tested the prototypes using semistructured telephone interviews with a purposive sample of Canadian healthcare managers and policy makers.

Results:Our review of the 223 HTAs identified that: (i) 44 HTAs addressed health system arrangements (20 percent); (ii) 205 incorporated a systematic review (92 percent), whereas only 12 incorporated a sociopolitical assessment using explicit methods (5 percent); and (iii) 50 contained context-specific actionable messages (22 percent). Our interviews identified significant support for both the general idea of an HTA summary and the prototype's specific elements, but mixed views about using peer assessments of relevance.

Conclusions:Those involved in supporting the use of HTAs in policy making about health systems may wish to produce structured decision-relevant summaries for their systematic review-containing HTAs to increase the prospects for their HTAs being used outside the jurisdiction for which they were produced.

Systems to support health technology assessment (HTA) in Member States of the European Union with limited institutionalization of HTA

Objectives: The aim of this study was to support health technology assessment (HTA) capacity building in Member States of the European Union with limited experience or without institutionalized HTA. The main output is a Handbook on HTA Capacity Building.

Methods: The methods used were worldwide surveys of (i) HTA organizations, (ii) information management units, and (iii) HTA educational programs. The results of two surveys (i & ii) were combined with expert opinion to produce the Handbook on HTA Capacity Building.

Results:Survey of HTA organizations (n = 41, response rate 35 percent). Most of the organizations were established by the government (61 percent), and all were not-for-profit. Working on HTA (80.5 percent) and doing research (63.4 percent) were the main lines of activity. Survey on information management units (n = 23, response rate 23 percent). Most (74.2 percent) of the responding HTA agencies reported having personnel dedicated to HTA information services. Survey on HTA educational programs (n = 48, response rate 60 percent). In total, nine Master of Science (MSc) programs were identified (three MSc in HTA and six MSc in HTA-related areas). Handbook on HTA Capacity Building. A group of twenty experts from thirteen countries developed the handbook. It consists of nine chapters focusing on HTA institutional development (structural setup, work processes, and visibility).

Conclusions: Setting up organizational structures and establishing effective HTA programs that guide key policy decisions is a challenging task. There are no standard models or pathways. “One size fits all” is not a useful principle because of the wide systemic and cultural differences between countries. The Handbook on HTA Capacity Building includes approaches for overall institutional development, especially in formulating objectives, setting up structures, and defining work processes.

Ethics in Canadian health technology assessment: A descriptive review

Background: Despite the mandate to examine the medical, ethical, and economic implications of the development and use of health technology, health technology assessment (HTA) reports often emphasize the epidemiologic and economic aspects, and omit ethical considerations. This study examines both whether and how ethical issues are incorporated into HTA.

Objectives: We aim to (i) review a set of Canadian HTA reports for ethics content, (ii) describe the strategies used to incorporate ethically relevant information into HTA, and (iii) determine the presence of implicit ethical issues in a sample of HTA reports.

Methods: Descriptive and qualitative content analysis of 608 HTA reports produced by six Canadian HTA agencies from January 1997 to December 2006.

Results: We found that (i) a minority (17 percent) of Canadian HTA reports addressed ethical issues, (ii) secondary research predominates while primary analysis is rare, (iii) implicit ethical issues are present in HTA reports that do not purport to address ethics.

Conclusions: Canadian HTA reports rarely explicitly, and then only superficially, address ethics, though implicit ethical issues abound.

Methods, procedures, and contextual characteristics of health technology assessment and health policy decision making: comparison of health technology assessment agencies in Germany, United Kingdom, France, and Sweden

OBJECTIVES

The objectives of this study were (i) to develop a systematic framework for describing and comparing different features of health technology assessment (HTA) agencies, (ii) to identify and describe similarities and differences between the agencies, and (iii) to draw conclusions both for producers and users of HTA in research, policy, and practice.

METHODS

We performed a systematic literature search, added information from HTA agencies, and developed a conceptual framework comprising eight main domains: organization, scope, processes, methods, dissemination, decision, implementation, and impact. We grouped relevant items of these domains in an evidence table and chose five HTA agencies to test our framework: DAHTA@DIMDI, HAS, IQWiG, NICE, and SBU. Item and domain similarity was assessed using the percentage of identical characteristics in pairwise comparisons across agencies. RESULTS were interpreted across agencies by demonstrating similarities and differences.

RESULTS

Based on 306 included documents, we identified 90 characteristics of eight main domains appropriate for our framework. After applying the framework to the five agencies, we were able to show 40 percent similarities in "dissemination," 38 percent in "scope," 35 percent in "organization," 29 percent in "methods," 26 percent in "processes," 23 percent in "impact," 19 percent in "decision," and 17 percent in "implementation."

CONCLUSION

We found considerably more differences than similarities of HTA features across agencies and countries. Our framework and comparison provides insights and clarification into the need for harmonization. Our findings could serve as descriptive database facilitating communication between producers and users.

Defining products for a new health technology assessment agency in Madrid, Spain: A survey of decision makers

Objectives:The aim of this study was to explore the needs and requirements of decision makers in our regional healthcare system for health technology assessment (HTA) products to support portfolio development planning for a new HTA agency in Madrid, Spain.

Methods:A Delphi study was conducted during 2003. Questionnaires were developed based on a review of products and services offered by other agency members of the International Network of Agencies for Health Technology Assessment, and included preference and prioritization questions to evaluate twenty-two different products and services. The initial Delphi panel involved eighty-seven experts from twenty-one public hospitals, eleven primary healthcare centers, six private hospitals, and eight departments of the Regional Ministry of Health of the Community of Madrid.

Results:The global participation rate was 83.9 percent. Ten of the twenty-two possible products were rated of high interest by more than 80 percent of respondents. Important differences in preferences and priorities were detected across different settings. Public hospitals and primary healthcare centers shared a more “micro” perspective, preferring classic technology-centered HTA products, whereas private hospitals and Ministry representatives demanded more “macro” products and services such as organizational model and information system assessments.

Conclusions:The high participation rate supports the representativeness of the results for our regional context. The strategic development of an HTA portfolio based on decision makers’ needs and requirements as identified in this type of exercise should help achieve a better impact on policy development and decision making.

Exploring policy-makers’ perspectives on disinvestment from ineffective healthcare practices

Objectives:Many existing healthcare interventions diffused before modern evidence-based standards of clinical- and cost-effectiveness. Disinvestment from ineffective or inappropriately applied practices is growing as a priority for international health policy, both for improved quality of care and sustainability of resource allocation. Australian policy stakeholders were canvassed to assess their perspectives on the challenges and the nature of disinvestment.

Methods:Senior health policy stakeholders from Australia were criterion and snow-ball sampled (to identify opinion leaders). Participants were primed with a potential disinvestment case study and took part in individual semistructured interviews that focused on mechanisms and challenges within health policy to support disinvestment. Interviews were taped and transcribed for thematic analysis. Participant comments were de-identified.

Results:Ten stakeholders were interviewed before saturation was reached. Three primary themes were identified. (i) The current focus on assessment of new and emerging health technologies/practices and lack of attention toward existing practices is due to resource limitations and methodological complexity. Participants considered a parallel model to that of Australia's current assessment process for new medical technologies is best-positioned to facilitate disinvestment. (ii) To advance the disinvestment agenda requires an explicit focus on the potential for cost-savings coupled with improved quality of care. (iii) Support (financial and collaborative) is needed for research advancement in the methodological underpinnings associated with health technology assessment and for disinvestment specifically.

Conclusions:In this exploratory study, stakeholders support the notion that systematic policy approaches to disinvestment will improve equity, efficiency, quality, and safety of health care, as well as sustainability of resource allocation.

Challenges in Australian policy processes for disinvestment from existing, ineffective health care practices

BACKGROUND

Internationally, many health care interventions were diffused prior to the standard use of assessments of safety, effectiveness and cost-effectiveness. Disinvestment from ineffective or inappropriately applied practices is a growing priority for health care systems for reasons of improved quality of care and sustainability of resource allocation. In this paper we examine key challenges for disinvestment from these interventions and explore potential policy-related avenues to advance a disinvestment agenda.

RESULTS

We examine five key challenges in the area of policy driven disinvestment: 1) lack of resources to support disinvestment policy mechanisms; 2) lack of reliable administrative mechanisms to identify and prioritise technologies and/or practices with uncertain clinical and cost-effectiveness; 3) political, clinical and social challenges to removing an established technology or practice; 4) lack of published studies with evidence demonstrating that existing technologies/practices provide little or no benefit (highlighting complexity of design) and; 5) inadequate resources to support a research agenda to advance disinvestment methods. Partnerships are required to involve government, professional colleges and relevant stakeholder groups to put disinvestment on the agenda. Such partnerships could foster awareness raising, collaboration and improved health outcome data generation and reporting. Dedicated funds and distinct processes could be established within the Medical Services Advisory Committee and Pharmaceutical Benefits Advisory Committee to, a) identify technologies and practices for which there is relative uncertainty that could be the basis for disinvestment analysis, and b) conduct disinvestment assessments of selected item(s) to address existing practices in an analogous manner to the current focus on new and emerging technology. Finally, dedicated funding and cross-disciplinary collaboration is necessary to build health services and policy research capacity, with a focus on advancing disinvestment research methodologies and decision support tools.

CONCLUSION

The potential over-utilisation of less than effective clinical practices and the potential under-utilisation of effective clinical practices not only result in less than optimal care but also fragmented, inefficient and unsustainable resource allocation. Systematic policy approaches to disinvestment will improve equity, efficiency, quality and safety of care, as well as sustainability of resource allocation.

Recommendations in health technology assessments worldwide

Objectives:The objective of this study is to analyze statistically the possible determinants and implications of including or not including recommendations in health technology assessments (HTAs).

Methods:A sample of 433 HTAs published by eleven leading institutions or agencies in nine countries was reviewed and analyzed statistically by multiple logistic regression.

Results:The extent of policy and research recommendations in HTAs varies greatly from country to country. The content and scope of HTAs have some impact on recommendations. Extensive assessment of economic and organizational aspects increases the likelihood of including policy recommendations. Extensive assessment of technological and patient aspects increases the likelihood of including research recommendations, whereas extensive focus on economic aspects is negatively related to research recommendations. The most striking result is that the use of external partners for assessment increases the likelihood of including research recommendations in HTAs but not policy recommendations.

Conclusions:HTA commissioners, agencies, institutions, and funding authorities need to be aware of the consequences of the choices they make in advance of assessing health technologies. Outsourcing HTA to external partners suggests a greater likelihood of being told that “more research is needed.” The scope and content of HTAs has an impact on the type of recommendations, and country-specific preferences are strong predictors of recommendations in HTAs.

Selection of new health technologies for assessment aimed at informing decision making: A survey among horizon scanning systems

Objectives:Uncertainty is pervasive in decision making on new health technologies; therefore, some countries have put systems in place to support decision makers with timely information. An important, but as yet undocumented, determinant of the potential value for decision making of these so-called horizon scanning systems (HSSs) is how the most significant health technologies are selected.

Methods:All thirteen member organizations of EuroScan, a collaborative network for HSSs, were surveyed and interviewed on how they prioritize technologies for assessment.

Results:The majority of HSSs directly serves a customer. Some customers actively request early assessments of new health technologies, thereby diminishing the need for priority setting for the HSSs. All systems express a concern to miss an important technology and/or to select an unimportant technology. Almost all HSSs use explicit selection criteria, but these criteria hardly ever are operationalized. The number of criteria used varies, but costs and health benefit of the technology are always taken into account. The process of reaching a final decision is implicit, undocumented in all but one system, and is based on agreement by consensus.

Conclusions:The process of making the final decision on which technologies to assess can be improved by applying existing criteria more consistently and transparently. Current practice does not safeguard against missing an important technology. This finding is probably most important to act upon for systems with customers that do not actively request assessment of specific technologies.

What influences the choice of assessment methods in health technology assessments? Statistical analysis of international health technology assessments from 1989 to 2002

Objectives:Health technology assessment (HTA) has been used as input in decision making worldwide for more than 25 years. However, no uniform definition of HTA or agreement on assessment methods exists, leaving open the question of what influences the choice of assessment methods in HTAs. The objective of this study is to analyze statistically a possible relationship between methods of assessment used in practical HTAs, type of assessed technology, type of assessors, and year of publication.

Methods:A sample of 433 HTAs published by eleven leading institutions or agencies in nine countries was reviewed and analyzed by multiple logistic regression.

Results:The study shows that outsourcing of HTA reports to external partners is associated with a higher likelihood of using assessment methods, such as meta-analysis, surveys, economic evaluations, and randomized controlled trials; and with a lower likelihood of using assessment methods, such as literature reviews and “other methods.” The year of publication was statistically related to the inclusion of economic evaluations and shows a decreasing likelihood during the year span. The type of assessed technology was related to economic evaluations with a decreasing likelihood, to surveys, and to “other methods” with a decreasing likelihood when pharmaceuticals were the assessed type of technology.

Conclusions:During the period from 1989 to 2002, no major developments in assessment methods used in practical HTAs were shown statistically in a sample of 433 HTAs worldwide. Outsourcing to external assessors has a statistically significant influence on choice of assessment methods.