Practical tools and methods for health technology assessment in Europe: Structures, methodologies, and tools developed by the European network for Health Technology Assessment, EUnetHTA

Health Technology Assessment of Vaccines in Italy: History and Review of Applications

Background/Objectives. Many vaccines have been developed in recent decades, and many more will be available in the future. When new safe and effective vaccines are available, decision-makers must extensively assess them before including them in the national immunization plan and issuing recommendations. The Health Technology Assessment (HTA) could be an objective, transparent, and comprehensive approach to guiding the decision-making process for the use of vaccines. Objectives and Methods. The aim of this study was to review the indications for HTA use contained in Italian institutional documents on vaccination, namely the National Immunization Plans (NIPs) and available full Italian HTA reports on vaccines, assessing their availability at the time of national recommendations' introductions. Results. HTA has been recognised as an eligible approach to deciding upon the introduction of vaccines through the NIPs of 2012-2014 and 2017-2019, and the last NIP, of 2023-2025, highlights the lack of funding dedicated to the production of independent HTA reports that can be used for issuing recommendations. In 2007-2023, twenty full HTA reports on vaccines were published in Italy: eight reports on influenza vaccines, five on Human Papilloma Virus (HPV), three each on meningococcal and pneumococcal vaccines, and one on rotavirus vaccine. HTA was applied with different purposes, namely the evaluation of new vaccines or their re-assessment, but it was not always timely with respect to both the marketing authorisation and the issuing of national recommendations for use. Conclusions. As HTA can be considered the best tool to disentangle the overall value of vaccines, it would be desirable for it to be used more and more to provide the evidence for efficient resource use. This calls for action to improve the transfer of HTA results to decision-makers, to try to fill the gap between research and decision and foster evidence-based recommendations.

Knowledge management tools and mechanisms for evidence-informed decision-making in the WHO European Region: a scoping review

BACKGROUND

Knowledge management (KM) emerged as a strategy to promote evidence-informed decision-making. This scoping review aims to map existing KM tools and mechanisms used to promote evidence-informed health decision-making in the WHO European Region and identify knowledge gaps.

METHODS

Following the Joanna Briggs Institute (JBI) guidance for conducting scoping reviews, we searched Medline, PubMed, EMBASE, the Cochrane library, and Open Grey. We conducted a descriptive analysis of the general characteristics of the included papers and conducted narrative analysis of the included studies and categorized studies according to KM type and phase.

RESULTS

Out of 9541 citations identified, we included 141 studies. The KM tools mostly assessed are evidence networks, surveillance tools, observatories, data platforms and registries, with most examining KM tools in high-income countries of the WHO European region. Findings suggest that KM tools can identify health problems, inform health planning and resource allocation, increase the use of evidence by policymakers and stimulate policy discussion.

CONCLUSION

Policymakers and funding agencies are called to support capacity-building activities, and future studies to strengthen KM in the WHO European region particularly in Eastern Europe and Central Asia. An updated over-arching strategy to coordinate KM activities in the WHO European region will be useful in these efforts.

Considerations for transferability of health technology assessments: a scoping review of tools, methods, and practices

Health technology assessment (HTA) is commonly used to guide evidence-informed decisions to optimize resource use, prioritize policies, and support countries to achieve universal health coverage. Producing HTAs requires time, scientific expertise, and political commitment, but these are not available in all settings - especially in low- and middle-income countries (LMIC) where HTA processes may be less institutionalized. Transferring and adapting existing HTAs to local settings may offer a solution while reducing duplication efforts. This scoping review aims to provide an overview of tools, methods, approaches, and considerations which can aid HTA transfers. We systematically searched (from 2005 to 2020) six databases and, using predefined inclusion criteria, included twenty-two studies. Data extraction followed a structured process, while synthesis was more iterative. We identified a common approach for HTA transfers. It follows the de novo process of undertaking original HTAs, but with additional steps to assess relevance (applicability), quality, and transferability, as well as steps to adapt parameters where necessary. The EUnetHTA Adaptation Toolkit was the only tool that provided guidance for adapting multiple HTA domains. Other tools were specific to systematic reviews (n = 1) or economic evaluations (n = 12), where one provided guidance for systematic reviews of economic evaluations. Eight papers reported transferring an HTA, with only one transferring to an LMIC. Finally, we reported issues that may facilitate or hinder transferability. In conclusion, we identified fourteen transfer approaches in the form of guidance or checklists, but harmonized and pragmatic guidance for HTA transfers to suit settings with limited HTA capacity seems warranted.

GoTO: A Process-Navigation Tool for Telehealth and -Care Solutions, Designed to Ensure an Efficient Trajectory from Goal Setting to Outcome Evaluation

Objectives: The digital transformation of the healthcare sector involves the procurement and implementation of new health technologies, which will likely be a challenge to healthcare providers who are not part of large organizations. In response to the needs of small and middle-sized health and care organizations, we have developed a process navigator to guide providers of healthcare through the processes of innovation, the procurement of mature products, and their implementation in telehealth and telecare projects. Methods: A narrative overview identified health-technology-assessment-inspired models. Conversations with national and international colleagues identified project and implementation models. The origin of the included models was identified, and relevant articles were referred to to describe the essential principles, including the nature of stakeholder involvement and the evaluation processes when appropriate. Based on the inputs, we proposed the process navigator GoTO. Results: Six health-technology-assessment-inspired models, six project models, one implementation model, and one innovation model were identified and informed the creation of the GoTO process navigator. The navigator consists of four parts: inception (eight steps); materialization (three tracks, depending on the maturity of the planned solution); implementation (five steps); and the final assessment and evaluation. Conclusion: The GoTO process navigator is an intuitive guide for innovation, procurement, and implementation in telehealth and -care. The GoTo navigator can assist providers of digital health and care services throughout the process from the initial identification of goals to the final evaluation of outcomes.

Challenges and Opportunities for Companies to Build HTA/Payer Perspectives Into Drug Development Through the Use of a Dynamic Target Product Profile

Background: The target product profile (TPP) outlines the desired profile of a target product aimed at a particular disease and is used by companies to plan clinical development. Considering the increasing importance of health technology assessment (HTA) in informing reimbursement decisions, a robust TPP needs to be built to address HTA needs, to guide an integrated evidence generation plan that will support HTA submissions. This study assessed current practices and experiences of companies in building HTA considerations into TPP development.

Methods: An opinion survey was designed and conducted in 2019, as a cross-sectional questionnaire consisting of multiple-choice questions. The questionnaire provided a qualitative assessment of companies’ strategies and experiences in building HTA considerations into the TPP. Eligible survey participants were the senior management of Global HTA/Market Access Departments at 18 top international pharmaceutical companies.

Results: 11 companies responded to the survey. All companies included HTA requirements in TPP development, but the timing and process varied. The key focus of HTA input related to health problems and treatment pathways, clinical efficacy/effectiveness, and safety. Variance of HTA methods and different value frameworks were identified as a challenge for development plans. Stakeholder engagement, such as HTA scientific advice, was used to pressure test the TPP.

Conclusion: This research provides insight into current practice and potential opportunities for value-based drug development. It demonstrates the evolution of the TPP to encompass HTA requirements and suggests that the TPP could have a role as an iterative communication tool for use with HTA agencies to enhance an integrated evidence generation plan.

Towards compatibility of EUnetHTA JCA methodology and German HTA: a systematic comparison and recommendations from an industry perspective

OBJECTIVE

The transferability of the EU joint clinical assessment (JCA) reports for pharmaceuticals for the German benefit assessment was evaluated by systematically comparing EU JCA and German clinical assessments (CA) based on established assessment elements for HTA and assessing the potential impact of differences on Federal Joint Committee (Gemeinsamer Bundesausschuss, G-BA) ability to derive the therapeutic added value.

METHODS

Identification of all pharmaceuticals undergoing both, EU JCA and German CA between January 2016-June 2020. Qualitative review and data extraction from the assessments, assessment of methodological differences using a hierarchical model. Recommendations for harmonisation were developed and consented with pharmaceutical industry stakeholders.

RESULTS

Differences with potentially major impact: (1) View on differing treatment algorithms and definition of corresponding subpopulations/respective comparators. (2) Clinical relevance of surrogate/intermediate endpoints. Inclusion of different/surrogate morbidity endpoints resulting in different relative effectiveness conclusions. (3) Tolerance of study interventions not used according to marketing authorisation. (4) Different operationalisation and/or weighting of individual safety endpoints leading to differing relative safety conclusions. Differences with potentially minor impact: (1) Disagreement in risk of bias assessment for overall survival and its robustness against study limitations. (2) Use of patient-reported outcome symptom scales as measurements for health-related quality of life instruments.

CONCLUSION

While many synergies between EU JCA and German CA exist, we identified several aspects in HTA methodology that would benefit of harmonisation and ensure the transferability of future EU JCA to the German HTA process without duplicated evaluation requirements. For those, a set of recommendations was developed.

How can we discover the most valuable types of big data and artificial intelligence-based solutions? A methodology for the efficient development of the underlying analytics that improve care

BACKGROUND

Much has been invested in big data and artificial intelligence-based solutions for healthcare. However, few applications have been implemented in clinical practice. Early economic evaluations can help to improve decision-making by developers of analytics underlying these solutions aiming to increase the likelihood of successful implementation, but recommendations about their use are lacking. The aim of this study was to develop and apply a framework that positions best practice methods for economic evaluations alongside development of analytics, thereby enabling developers to identify barriers to success and to select analytics worth further investments.

METHODS

The framework was developed using literature, recommendations for economic evaluations and by applying the framework to use cases (chronic lymphocytic leukaemia (CLL), intensive care, diabetes). First, the feasibility of developing clinically relevant analytics was assessed and critical barriers to successful development and implementation identified. Economic evaluations were then used to determine critical thresholds and guide investment decisions.

RESULTS

When using the framework to assist decision-making of developers of analytics, continuing development was not always feasible or worthwhile. Developing analytics for progressive CLL and diabetes was clinically relevant but not feasible with the data available. Alternatively, developing analytics for newly diagnosed CLL patients was feasible but continuing development was not considered worthwhile because the high drug costs made it economically unattractive for potential users. Alternatively, in the intensive care unit, analytics reduced mortality and per-patient costs when used to identify infections (- 0.5%, - €886) and to improve patient-ventilator interaction (- 3%, - €264). Both analytics have the potential to save money but the potential benefits of analytics that identify infections strongly depend on infection rate; a higher rate implies greater cost-savings.

CONCLUSIONS

We present a framework that stimulates efficiency of development of analytics for big data and artificial intelligence-based solutions by selecting those applications of analytics for which development is feasible and worthwhile. For these applications, results from early economic evaluations can be used to guide investment decisions and identify critical requirements.

Developing a quality management system for the European Network for Health Technology Assessment (EUnetHTA): toward European HTA collaboration

OBJECTIVES

The European Network for Health Technology Assessment (EUnetHTA) was established in 2006 and comprises over eighty organizations from thirty European countries. In its fifth project phase (Joint Action 3), EUnetHTA set up a quality management system (QMS) to improve the efficiency and standardization of joint work. This article presents EUnetHTA's new QMS and outlines experiences and challenges during its implementation.

METHODS

Several working groups defined processes and methods to support assessment teams in creating high-quality assessment reports. Existing guidelines, templates, and tools were refined and missing parts were newly created and integrated into the new QMS framework. EUnetHTA has contributed to Health Technology Assessment (HTA) capacity building through training and knowledge sharing. Continuous evaluation helped to identify gaps and shortcomings in processes and structures.

RESULTS

Based on a common quality management concept and defined development and revision procedures, twenty-seven partner organizations jointly developed and maintained around forty standard operating procedures and other components of the QMS. All outputs were incorporated into a web-based platform, the EUnetHTA Companion Guide, which was launched in May 2018. Concerted efforts of working groups were required to ensure consistency and avoid duplication.

CONCLUSIONS

With the establishment of a QMS for jointly produced assessment reports, EUnetHTA has taken a significant step toward a sustainable model for scientific and technical collaboration within European HTA. However, the definition of processes and methods meeting the numerous requirements of healthcare systems across Europe remains an ongoing and challenging task.

Benchmarking health technology assessment agencies-methodological challenges and recommendations

OBJECTIVES

The objectives of the study were to establish a benchmarking tool to collect metrics to enable increased clarity regarding the differences and similarities across health technology assessment (HTA) agencies, to assess performance within and across HTA agencies, identify areas in the HTA processes in which time is spent and to enable ongoing performance improvement.

METHODS

Common steps and milestones in the HTA process were identified for meaningful benchmarking among agencies. A benchmarking tool consisting of eighty-six questions providing information on HTA agency organizational aspects and information on individual new medicine review timelines and outcomes was developed with the input of HTA agencies and validated in a pilot study. Data on 109 HTA reviews from five HTA agencies were analyzed to demonstrate the utility of this tool.

RESULTS

This study developed an HTA benchmarking methodology, comparative metrics showed considerable differences among the median timelines from assessment and appraisal to final HTA recommendation for the five agencies included in this analysis; these results were interpreted in conjunction with agency characteristics.

CONCLUSIONS

It is feasible to find consensus among HTA agencies regarding the common milestones of the review process to map jurisdiction-specific processes against agreed metrics. Data on characteristics of agencies such as their scope and remit enabled results to be interpreted in the appropriate local context. This benchmarking tool has promising potential utility to improve the transparency of the review process and to facilitate both quality assurance and performance improvement in HTA agencies.

A New Decision Model Approach for Health Technology Assessment and A Case Study for Dialysis Alternatives in Turkey

Background: This paper presents a generic Multi-Criteria Decision Analysis (MCDA) model for Health Technology Assessment (HTA) decision-making, which can be applied to a wide range of HTA studies, regardless of the healthcare technology type under consideration. Methods: The HTA Core Model^® of EUnetHTA was chosen as a basis for the development of the MCDA model because of its common acceptance among European Union countries. Validation of MCDA4HTA was carried out by an application with the HTA study group of the Turkish Ministry of Health. The commitment of the decision-making group is completed via an online application of 10 different questionnaires. The Analytic Hierarchy Process (AHP) is used to determine the weights. Scores of the criteria in MCDA4HTA are gathered directly from the HTA report. The performance matrix in this application is run with fuzzy Technique for Order Preference by Similarity to Ideal Solution (TOPSIS), fuzzy Vise Kriterijumska Optimizacija I Kompromisno Resenje (VIKOR), and goal programming MCDA techniques. Results: Results for fuzzy VIKOR, fuzzy TOPSIS, and goal programming are 0.018, 0.309, and 0.191 for peritoneal dialysis and 0.978, 0.677, and 0.327 for hemodialysis, respectively. Conclusions: Peritoneal dialysis is found to be the best choice under the given circumstances, despite its higher costs to society. As an integrated decision-making model for HTA, MCDA4HTA supports both evidence-based decision policy and the transparent commitment of multi-disciplinary stakeholders.

An empirical investigation of time-varying cost-effectiveness across the product life cycle

Cost-effectiveness is traditionally treated as a static estimate driven by clinical trial efficacy and drug price at launch. Prior studies suggest that cost-effectiveness varies over the drug's lifetime. We examined the impact of "learning by doing," one of the least studied drivers of changes in cost-effectiveness across the product life cycle. We combined time-series trends in effectiveness over time by cancer regimen using the Surveillance, Epidemiology, and End Results-Medicare database. We estimated the time-varying effects of treatments in colorectal and pancreatic cancer over their life cycle, including FOLFOX (leucovorin, 5-fluorouracil, and oxaliplatin) and gemcitabine, on survival of patients. Mean prices over time by strength and dosage form were calculated using historical wholesale acquisition costs. We found consistent downward trends in the mortality hazard ratios, which suggest that effectiveness improves over time. In the case of first-line FOLFOX for colorectal cancer, the implied incremental cost-effectiveness ratio based on the observational data fell from $610,000 per life year gained in 2004 to $27,000 per life year gained in 2011. Cost-effectiveness estimated at launch is unlikely to be representative of cost-effectiveness over the drug's lifetime. In the drugs studied, the impact of time-varying clinical effectiveness dominated the impact of changing prices overtime.

A Web-Based Communication Platform to Improve Home Care Services in Norway (DigiHelse): Pilot Study

Background

Home care service in Norway is struggling to meet the increasing demand for health care under restricted budget constraints, although one-fourth of municipal budgets are dedicated to health services. The integration of Web-based technology in at-home care is expected to enhance communication and patient involvement, increase efficiency and reduce cost. DigiHelse is a Web-based platform designed to reinforce home care service in Norway and is currently undergoing a development process to meet the predefined needs of the country’s municipalities. Some of the main features of the platform are digital messages between residents and the home care service, highlighting information on planned and completed visits, the opportunity to cancel visits, and notifications for completed visits.

Objective

This study aimed to test the usability and economic feasibility of adopting DigiHelse in four districts in Oslo by applying registry and behavioral data collected throughout a one-year pilot study. Early health technology assessment was used to estimate the potential future value of DigiHelse, including the predictive value of behavior data.

Methods

Outcome measures identified by stakeholder insights and scenario drafting in the project’s concept phase were used to assess potential socioeconomic benefits. Aggregated data were collected to assess changes in health consumption at baseline, and then 15 and 52 weeks after DigiHelse was implemented. The present value calculation was updated with data from four intervention groups and one control group. A quasi-experimental difference-in-difference design was applied to estimate the causal effect. Descriptive behavioral data from the digital platform was applied to assess the usability of the platform.

Results

Over the total study period (52 weeks), rates increased for all outcome estimates: the number of visits (rate ratio=1.04; P=.10), unnecessary trips (rate ratio=1.37; P=.26), and phone calls (rate ratio=1.24; P=.08). A significant gap was found between the estimated value of DigiHelse in the concept phase and after the one-year pilot. In the present pilot assessment, costs are expected to exceed potential savings by €67 million (US $75 million) over ten years, as compared to the corresponding concept estimates of a potential gain of €172.6 million (US $193.6 million). Interestingly, behavioral data from the digital platform revealed that only 3.55% (121/3405) of recipients actively used the platform after one year.

Conclusions

Behavioral data provides a valuable source for assessing usability. In this pilot study, the low adoption rate may, at least in part, explain the inability of DigiHelse to perform as expected. This study points to an early assessment of behavioral data as an opportunity to identify inefficiencies and direct digital development. For DigiHelse, insight into why the recipients in Oslo have not made greater use of the Web-based platform seems to be the next step in ensuring the right improvement measures for the home care service.

Scientific Development of HTA—A Proposal by the Health Technology Assessment International Scientific Development and Capacity Building Committee

Objectives

To report from the Scientific Development and Capacity Building Committee of Health Technology Assessment International (HTAi) on activities that are being undertaken within HTAi regarding the promotion of scientific rigor in the field of health technology assessment (HTA).

Methods

Retrieval of definitions of HTA that the SDCB committee considered reflective of the current practice of HTA, followed by a narrative synthesis of the core components of HTA.

Results

Several definitions of HTA have been provided, all sharing the notion that HTA is the formal, systematic, and transparent inquiry into the meaning and value, broadly defined, of health technologies, when used in specific patient populations.

Many frameworks and tools have been developed for assessing the quality of specific tasks that may be conducted in the context of HTA. Collating such frameworks and tools is likely to be helpful in developing standards and in providing guidance as to how the scientific quality of HTA may be secured. Two current trends in HTA were noted: a stronger health systems focus, and the need to involve stakeholders throughout the HTA process. A wider systems’ perspective requires that plausible alternative scenarios are being developed, and wide consultation of various stakeholders is a prerequisite to the development of such scenarios with data from various sources.

Conclusions

Current trends in HTA will lead to different demands on the HTA expert. The task of this emerging policy professional would be not just to provide technical information for problem-solving, but also to combine it with a new function of facilitating public deliberation and learning.

Hospital staff perspectives towards health technology assessment: data from a multidisciplinary survey

BACKGROUND

Technology adoption in hospitals is usually based on cost-effectiveness analysis, feasibility and potential success. Different countries have embraced a range of principles to accomplish an effective comprehensive process of health technology assessment (HTA). The aim of the study was to analyse the viewpoints and relative weight of technology-oriented hospital staff members toward the clinical, social, technological and economic aspects of HTA.

METHODS

Using a structured questionnaire, a survey was conducted among different professionals in an 850-bed hospital.

RESULTS

We revealed a range of viewpoints among hospital staff members according to their personal characteristics and professional standpoints. The clinical aspects of HTA were considered 'highly important' (HI) by most participants, especially the 'lifesaving' parameter. Similarly, the 'lack of effective alternative technology' was ranked HI by a high percentage of participants, independent of their profession. Economic aspects were ranked HI only by half of the participants, while social and technological aspects were ranked HI only by a relatively low percentage. Nurses added 'improving quality of life', 'increasing teamwork efficiency' and 'improving medical standards'. Allied health professionals focused on 'lack of effective alternative technologies' as a main argument for adoption of HTA, alongside increasing efficiency, budget savings and contribution to hospital reputation. Engineers emphasised the requirement of significant investment in infrastructure and increasing efficiency. Administrators ranked patient experience as HI. Interestingly, the high ranking of social aspects correlated with older responders, while junior staff ranked safety significantly higher.

CONCLUSIONS

A multi-perspective multidisciplinary approach would be beneficial for policy-makers at hospitals and even on a national scale in Israel.

Early assessment of innovation in a healthcare setting

OBJECTIVES

Early assessment can assist in allocating resources for innovation effectively and produce the most beneficial technology for an institution. The aim of the present study was to identify methods and discuss the analytical approaches applied for the early assessment of innovation in a healthcare setting.

METHODS

Knowledge synthesis based on a structured search (using the MEDLINE, Embase, and Cochrane databases) and thematic analysis was conducted. An analytical framework based on the stage of innovation (developmental, introduction, or early diffusion) was applied to assess whether methods vary according to stage. Themes (type of innovation, study, analysis, study design, method, and main target audience) were then decided among the authors. Identified methods and analysis were discussed according to the innovation stage.

RESULTS

A total of 1,064 articles matched the search strategy. Overall, thirty-nine articles matched the inclusion criteria. The use of methods has a tendency to change according to the stage of innovation. Stakeholder analysis was a prominent method in the innovation stages and particularly in the developmental stage, as the introduction and early diffusion stage has more availability of data and may apply more complex methods. Barriers to the identified methods were also discussed as all of the innovation stages suffered from lack of data and substantial uncertainty.

CONCLUSIONS

Although this review has identified applicable approaches for early assessment in different innovation stages, research is required regarding the value of the available data and methods and tools to enhance interactions between different parties at different stages of innovation.

European Collaboration in Health Technology Assessment (HTA): goals, methods and outcomes with specific focus on medical devices

The European Network for Health Technology Assessment (EUnetHTA) was founded to support efficient production and use of health technology assessments (HTAs) across Europe by reducing redundancies through collaboration. To facilitate collaboration, a range of practical tools, methods and process definitions were developed. The article describes when and how these tools and methods are used along the HTA process with specific focus on “other technologies”, that is medical devices and non-pharmaceutical procedures. EUnetHTA was able to deliver tangible achievements complying with its goals. The practical tools and the developed methods formed a basis for close collaboration among over 70 agencies at a European level. The activities of EUnetHTA laid a strong foundation for sustainable cooperation. In the long run, jointly produced assessments could realise economies of scale with improved quality, consistency and transparency for the health systems in Europe.

Comparing Use of Health Technology Assessment in Pharmaceutical Policy among Earlier and More Recent Adopters in the European Union

OBJECTIVES

To examine and compare the use of health technology assessment (HTA) for the reimbursement of new medicines in selected European Union member states with decades of experience in the use of HTA and in countries that have used it regularly since 2000.

METHODS

The selected countries were categorized into "earlier" adopters (group A: England, Germany, France, and Sweden) and more "recent" adopters (group B: Poland, Bulgaria, Hungary, and Romania). A systematic review of published literature was performed. The analysis and comparison of HTA procedures were done by using an analytical framework.

RESULTS

In all countries, the assessment criteria used include effectiveness, safety, relative effectiveness, and economic data. In group A countries, the main objectives are improving quality of care, ensuring equal access, and efficient use of resources. Group B countries have established HTA organizations with official guidelines but often seek the decisions of other developed countries. They place considerable emphasis on the budget impact of new therapies, and HTA is also used as a cost estimation tool for state budgets.

CONCLUSIONS

HTA organizations have been developed dynamically not only in high-income countries but also in countries with limited resources. The experience and evolution of both can be used by countries that are in the dawn of creating an HTA organization.

Assessment of Health Information Technology Interventions in Evidence-Based Medicine: A Systematic Review by Adopting a Methodological Evaluation Framework

BACKGROUND

The application of Health Information Technologies (HITs) can be an effective way to advance medical research and health services provision. The two-fold objective of this work is to: (i) identify and review state-of-the-art HITs that facilitate the aims of evidence-based medicine and (ii) propose a methodology for HIT assessment.

METHODS

The systematic review was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Furthermore, we consolidated existing knowledge in the field and proposed a Synthesis Framework for the Assessment of Health Information Technology (SF/HIT) in order to evaluate the joint use of Randomized Controlled Trials (RCTs) along with HITs in the field of evidence-based medicine.

RESULTS

55 articles met the inclusion criteria and refer to 51 (RCTs) published between 2008 and 2016. Significant improvements in healthcare through the use of HITs were observed in the findings of 31 out of 51 trials-60.8%. We also confirmed that RCTs are valuable tools for assessing the effectiveness, acceptability, safety, privacy, appropriateness, satisfaction, performance, usefulness and adherence.

CONCLUSIONS

To improve health service delivery, RCTs apply and exhibit formalization by providing measurable outputs. Towards this direction, we propose the SF/HIT as a framework which may help researchers to carry out appropriate evaluations and extend their studies.

DECISION-COMPONENTS OF NICE'S TECHNOLOGY APPRAISALS ASSESSMENT FRAMEWORK

Objectives: Value assessment frameworks have gained prominence recently in the context of U.S. healthcare. Such frameworks set out a series of factors that are considered in funding decisions. The UK's National Institute of Health and Care Excellence (NICE) is an established health technology assessment (HTA) agency. We present a novel application of text analysis that characterizes NICE's Technology Appraisals in the context of the newer assessment frameworks and present the results in a visual way.

Methods: A total of 243 documents of NICE's medicines guidance from 2007 to 2016 were analyzed. Text analysis was used to identify a hierarchical set of decision factors considered in the assessments. The frequency of decision factors stated in the documents was determined and their association with terms related to uncertainty. The results were incorporated into visual representations of hierarchical factors.

Results: We identified 125 decision factors, and hierarchically grouped these into eight domains: Clinical Effectiveness, Cost Effectiveness, Condition, Current Practice, Clinical Need, New Treatment, Studies, and Other Factors. Textual analysis showed all domains appeared consistently in the guidance documents. Many factors were commonly associated with terms relating to uncertainty. A series of visual representations was created.

Conclusions: This study reveals the complexity and consistency of NICE's decision-making processes and demonstrates that cost effectiveness is not the only decision-criteria. The study highlights the importance of processes and methodology that can take both quantitative and qualitative information into account. Visualizations can help effectively communicate this complex information during the decision-making process and subsequently to stakeholders.

ANALYSIS OF DUPLICATION AND TIMING OF HEALTH TECHNOLOGY ASSESSMENTS ON MEDICAL DEVICES IN EUROPE

OBJECTIVES

Strengthening efforts toward better collaboration plays a pivotal role in the assessment of medical devices to reduce overlap and save resources. This study explored the level of duplication in health technology assessments (HTA) of medical devices in Europe and their respective timing in order to identify areas for better collaboration.

METHODS

An analysis of European HTA reports of medical devices regarding overlaps in topics and timing in relation to market authorization was performed. We conducted a systematic search in the ADVANCE, Centre for Reviews and Dissemination, Syngerus, and POP databases, complemented by hand searching, to identify HTA reports published between 01/2003 and 07/2016 for a preselected cohort group of ten technologies. We analyzed the number of annual assessments per technology and evaluated activity patterns and timing in undertaking the HTA of the different institutes in Europe.

RESULTS

The results revealed the amount of duplication in the European HTA production: the number of reports per technology ranged from minimum seven to maximum twenty-two over a time-span of 13.5 years. HTA institutes perform assessments at a similar time range within 5 to 10 years following market authorization. The timing of the initial assessment in relation to the granting of the CE-mark varies according to the particular technology.

CONCLUSION

The findings suggest that efficient collaboration may help to save scarce resources and time of HTA institutes in Europe. Efficient collaboration as such needs to shift the focus beyond the time span of 1 year, and build on each's others work from previous assessments.

Health Technology Assessment capacity development in low- and middle-income countries: Experiences from the international units of HITAP and NICE

Health Technology Assessment (HTA) is policy research that aims to inform priority setting and resource allocation. HTA is increasingly recognized as a useful policy tool in low- and middle-income countries (LMICs), where there is a substantial need for evidence to guide Universal Health Coverage policies, such as benefit coverage, quality improvement interventions and quality standards, all of which aim at improving the efficiency and equity of the healthcare system. The Health Intervention and Technology Assessment Program (HITAP), Thailand, and the National Institute for Health and Care Excellence (NICE), UK, are national HTA organizations providing technical support to governments in LMICs to build up their priority setting capacity. This paper draws lessons from their capacity building programs in India, Colombia, Myanmar, the Philippines, and Vietnam. Such experiences suggest that it is not only technical capacity, for example analytical techniques for conducting economic evaluation, but also management, coordination and communication capacity that support the generation and use of HTA evidence in the respective settings. The learned lessons may help guide the development of HTA capacity in other LMICs.

PRIORITIES FOR HEALTH ECONOMIC METHODOLOGICAL RESEARCH: RESULTS OF AN EXPERT CONSULTATION

BACKGROUND

The importance of economic evaluation in decision making is growing with increasing budgetary pressures on health systems. Diverse economic evidence is available for a range of interventions across national contexts within Europe, but little attention has been given to identifying evidence gaps that, if filled, could contribute to more efficient allocation of resources. One objective of the Research Agenda for Health Economic Evaluation project is to determine the most important methodological evidence gaps for the ten highest burden conditions in the European Union (EU), and to suggest ways of filling these gaps.

METHODS

The highest burden conditions in the EU by Disability Adjusted Life Years were determined using the Global Burden of Disease study. Clinical interventions were identified for each condition based on published guidelines, and economic evaluations indexed in MEDLINE were mapped to each intervention. A panel of public health and health economics experts discussed the evidence during a workshop and identified evidence gaps.

RESULTS

The literature analysis contributed to identifying cross-cutting methodological and technical issues, which were considered by the expert panel to derive methodological research priorities.

CONCLUSIONS

The panel suggests a research agenda for health economics which incorporates the use of real-world evidence in the assessment of new and existing interventions; increased understanding of cost-effectiveness according to patient characteristics beyond the "-omics" approach to inform both investment and disinvestment decisions; methods for assessment of complex interventions; improved cross-talk between economic evaluations from health and other sectors; early health technology assessment; and standardized, transferable approaches to economic modeling.

Congruence between patient characteristics and interventions may partly explain medication adherence intervention effectiveness: an analysis of 190 randomized controlled trials from a Cochrane systematic review

OBJECTIVES

Due to the negative outcomes of medication nonadherence, interventions to improve adherence have been the focus of countless studies. The congruence between adherence-related patient characteristics and interventions may partly explain the variability of effectiveness in medication adherence studies. In their latest update of a Cochrane review reporting inconsistent effects of adherence interventions, the authors offered access to their database for subanalysis. We aimed to use this database to assess congruence between adherence-related patient characteristics and interventions and its association with intervention effects.

STUDY DESIGN AND SETTING

We developed a congruence score consisting of six features related to inclusion criteria, patient characteristics at baseline, and intervention design. Two independent raters extracted and scored items from the 190 studies available in the Cochrane database. We correlated overall congruence score and individual features with intervention effects regarding adherence and clinical outcomes using Kruskal-Wallis rank sum test and Fisher's exact test.

RESULTS

Interrater reliability for newly extracted data was almost perfect with a Cohen's Kappa of 0.92 [95% confidence interval (CI) = 0.89-0.94; P < 0.001]. Although present in only six studies, the inclusion of nonadherent patients was the single feature significantly associated with effective adherence interventions (P = 0.003). Moreover, effective adherence interventions were significantly associated with improved clinical outcomes (odds ratio = 6.0; 95% CI = 3.1-12.0; P < 0.0001). However, neither the overall congruence score nor any other individual feature (i.e., "determinants of nonadherence as inclusion criteria," "tailoring of interventions to the inclusion criteria," "reasons for nonadherence assessed at baseline," "adjustment of intervention to individual patient needs," and "theory-based interventions") was significantly associated with intervention effects.

CONCLUSION

The presence of only six studies that included nonadherent patients and the interdependency of this feature with the remaining five might preclude a conclusive assessment of congruence between patient characteristics and adherence interventions. In order to obtain clinical benefits from effective adherence interventions, we encourage researchers to focus on the inclusion of nonadherent patients.

Developing a decision support system to link health technology assessment (HTA) reports to the health system policies in Iran

The recent increase of 'Health Technology Assessment' (HTA)-related activities in Iran has necessitated the clarification of policy-making process based on the HTA reports. This study aimed to develop a Decision Support System (DSS) in order to adopt evidence-informed policies regarding health technologies in Iran. The study can be classified as Health Policy and Systems Research. A core panel of seven experts conducted two separate reviews of relevant literature for: 1- Determining the potential technology-related policies. 2- Listing the criteria influencing those policy decisions. The policies and criteria were separately discussed and subsequently rated for appropriateness and necessity during two expert meetings in 2013. In the next step, The 'Discrete Choice Experiment' (DCE) method was employed to develop the DSS for the final technology-related policies. Accordingly, the core panel members independently rated the appropriateness of each policy for 30 virtual technologies based on the random values assigned to all the criteria for each technology. The obtained data for each policy were separately analysed using stepwise regression model, resulting in a minimal set of independent and statistically significant criteria contributing in the experts' judgments about the appropriateness of that policy. The obtained regression coefficients were used as the relative weights of the different levels of the final criteria of any policy statement, shaping the decision support scoring tool for each policy. The study has outlined 64 policy decisions under 7 macro policy areas concerning a health technology. Also, 34 criteria used for making those policy decisions have been organized within a portfolio. DCE, using stepwise regression, resulted in 64 scoring tools shaping the DSS for all HTA-related policies. Both the results and methodology of the study may serve as a guide for policy makers (researchers), particularly in low and middle income countries, in developing decision aids for their own context-specific HTA-related policies.

Burden of Disease Study and Priority Setting in Korea: an Ethical Perspective

When thinking about priority setting in access to healthcare resources, decision-making requires that cost-effectiveness is balanced against medical ethics. The burden of disease has emerged as an important approach to the assessment of health needs for political decision-making. However, the disability adjusted life years approach hides conceptual and methodological issues regarding the claims and value of disabled people. In this article, we discuss ethical issues that are raised as a consequence of the introduction of evidence-based health policy, such as economic evidence, in establishing resource allocation priorities. In terms of ethical values in health priority setting in Korea, there is no reliable rationale for the judgment used in decision-making as well as for setting separate and distinct priorities for different government bodies. An important question, therefore, is which ethical values guiding the practice of decision-making should be reconciled with the economic evidence found in Korean healthcare. The health technology assessment core model from the European network for Health Technology Assessment (EUnetHTA) project is a good example of incorporating ethical values into decision-making. We suggest that a fair distribution of scarce healthcare resources in South Korea can be achieved by considering the ethical aspects of healthcare.

HTA Implementation Roadmap in Central and Eastern European Countries

The opportunity cost of inappropriate health policy decisions is greater in Central and Eastern European (CEE) compared with Western European (WE) countries because of poorer population health and more limited healthcare resources. Application of health technology assessment (HTA) prior to healthcare financing decisions can improve the allocative efficiency of scarce resources. However, few CEE countries have a clear roadmap for HTA implementation. Examples from high-income countries may not be directly relevant, as CEE countries cannot allocate so much financial and human resources for substantiating policy decisions with evidence. Our objective was to describe the main HTA implementation scenarios in CEE countries and summarize the most important questions related to capacity building, financing HTA research, process and organizational structure for HTA, standardization of HTA methodology, use of local data, scope of mandatory HTA, decision criteria, and international collaboration in HTA. Although HTA implementation strategies from the region can be relevant examples for other CEE countries with similar cultural environment and economic status, HTA roadmaps are not still fully transferable without taking into account country-specific aspects, such as country size, gross domestic product per capita, major social values, public health priorities, and fragmentation of healthcare financing.

Planned and ongoing projects (pop) database: development and results

OBJECTIVES

The aim of this study was to present the development, structure and results of a database on planned and ongoing health technology assessment (HTA) projects (POP Database) in Europe.

METHODS

The POP Database (POP DB) was set up in an iterative process from a basic Excel sheet to a multifunctional electronic online database. The functionalities, such as the search terminology, the procedures to fill and update the database, the access rules to enter the database, as well as the maintenance roles, were defined in a multistep participatory feedback loop with EUnetHTA Partners.

RESULTS

The POP Database has become an online database that hosts not only the titles and MeSH categorizations, but also some basic information on status and contact details about the listed projects of EUnetHTA Partners. Currently, it stores more than 1,200 planned, ongoing or recently published projects of forty-three EUnetHTA Partners from twenty-four countries. Because the POP Database aims to facilitate collaboration, it also provides a matching system to assist in identifying similar projects. Overall, more than 10 percent of the projects in the database are identical both in terms of pathology (indication or disease) and technology (drug, medical device, intervention). In addition, approximately 30 percent of the projects are similar, meaning that they have at least some overlap in content.

CONCLUSIONS

Although the POP DB is successful concerning regular updates of most national HTA agencies within EUnetHTA, little is known about its actual effects on collaborations in Europe. Moreover, many non-nationally nominated HTA producing agencies neither have access to the POP DB nor can share their projects.

EVIDENCE REQUIREMENTS FOR REIMBURSEMENT OF PHARMACEUTICALS ACROSS EUROPE

Objectives: The objective of this study was to compare evidence requirements for health technology assessment of pharmaceuticals by national agencies across Europe responsible for reimbursement decisions focusing specifically on relative effectiveness assessment.

Methods: Evidence requirements from thirty-three European countries were requested and twenty-nine national agencies provided documents to review. Data were extracted from national documents (manufacturer's submission templates and associated guidance) into a purpose-made framework with categories covering information about the health condition, the technology, clinical effectiveness and safety.

Results: The level of detail in the required evidence varies considerably across countries. Some countries include specific questions while others request information under general headings. Some countries include all information in a single document, which may or may not include guidance on how to complete the template. Others have specific guidance documents or methods and process manuals that help with the completion of the submission templates. Despite differences in quantity and detail, the content of the evidence requirements is broadly similar. All countries ask for information on the health technology, target disease, and clinical effectiveness and safety. However, one country only requests clinical effectiveness information as part of cost-effectiveness analyses. We found twenty-six evidence requirements for which generic answers may apply across borders and nineteen in which countries requested nationally specific information.

Conclusions: This work suggests that it would be possible to put together a minimum set of evidence requirements for HTA to support reimbursement decisions across Europe which could facilitate collaboration between jurisdictions.

The vulnerable aspects of application of "Health Technology Assessment"

Many countries in the world have tried to examine the possible methods for import and logical use of health technologies to manage their budgets on one hand and to prevent the entry of uncertain, inefficient, and insecure technologies on the other hand (1). The “health technology assessment” (HTA) is one of the dominant methods in most developed countries (2). HTA is a multidisciplinary field which studies the medical, social, and ethical aspects, as well as economic outcomes of production, diffusion, and application of health technologies (3).

DEVELOPING THE HTA CORE MODEL FOR THE ONLINE ENVIRONMENT

Background: A framework for collaborative production and sharing of HTA information, the HTA Core Model, was originally developed within EUnetHTA in 2006–08. In this paper, we describe the further development of the Model to allow implementation and utilization of the Model online. The aim was to capture a generic HTA process that would allow effective use of the HTA Core Model and resulting HTA information while at the same time not interfering with HTA agencies' internal processes.

Methods: The work was coordinated by a development team in Finland, supported by an international expert group. Two pilot testing rounds were organized among EUnetHTA agencies and two extensive core HTA projects tested the tool in a real setting. The final work was also formally validated by a group of HTA agencies.

Results: The HTA Core Model Online—available at http://www.corehta.info—is a web site hosting a) a tool to allow electronic utilization of the HTA Core Model and b) a database of produced HTA information. While access to the HTA information is free to all, the production features are currently available to EUnetHTA member agencies only. A policy was crafted to steer the use of the Model and produced information.

Conclusions: We have successfully enabled electronic use of the HTA Core Model and agreed on a policy for its utilization. The system is already being used in subsequent HTA projects within EUnetHTA Joint Action 2. Identified shortcomings and further needs will be addressed in subsequent development.

From adaptive licensing to adaptive pathways: delivering a flexible life-span approach to bring new drugs to patients

The concept of adaptive licensing (AL) has met with considerable interest. Yet some remain skeptical about its feasibility. Others argue that the focus and name of AL should be broadened. Against this background of ongoing debate, we examine the environmental changes that will likely make adaptive pathways the preferred approach in the future. The key drivers include: growing patient demand for timely access to promising therapies, emerging science leading to fragmentation of treatment populations, rising payer influence on product accessibility, and pressure on pharma/investors to ensure sustainability of drug development. We also discuss a number of environmental changes that will enable an adaptive paradigm. A life-span approach to bringing innovation to patients is expected to help address the perceived access vs. evidence trade-off, help de-risk drug development, and lead to better outcomes for patients.

[Health technology assessment (HTA). Developments in healthcare and potential for radiology]

CLINICAL/METHODICAL ISSUE

Cost-intensive measures and procedures, such as also employed in radiology, have far-reaching economic implications in respect to increasing expenditure with limited resources.

STANDARD RADIOLOGICAL METHODS

Health technology assessment (HTA) describes the systematic evaluation of medical procedures and technologies which in recent years has been introduced by many countries into healthcare politics.

ASSESSMENT

In many cases HTA analyses can be directly implemented into practice as shown by the examples given in this article; however, in the current form of HTA the practical implementation for radiology often presents the problem that the cost-benefit ratio does not yet have a comprehensive view in the HTA report but is limited to a subsection, e.g. current costs versus sensitivity of a method. Since its inception radiology has had a high power of innovation and new developments will also substantially determine the future years. These procedures must not only be evaluated with respect to feasibility but also in the sense of the HTA in the total concept.

PRACTICAL RECOMMENDATIONS

In radiology there are also a large number of possibilities for radiologists not only as passive consumers of HTA reports but also to become active participants in this process, an opportunity which should be taken advantage of.

WHAT’S IN A NAME? DEVELOPING DEFINITIONS FOR COMMON HEALTH TECHNOLOGY ASSESSMENT PRODUCT TYPES OF THE INTERNATIONAL NETWORK OF AGENCIES FOR HEALTH TECHNOLOGY ASSESSMENT (INAHTA)

Objectives: A mapping exercise was undertaken to determine how HTA is being described and conducted across the International Network of Agencies for Health Technology Assessment (INAHTA), with the aim of harmonizing terminologies and approaches.

Methods: Three progressive surveys were undertaken. In 2010, INAHTA agencies were asked to provide details on all of their HTA products. In 2013, additional information was sought on key methodological characteristics of five of the most common HTA product types. Subsequently, final agreement was sought on three proposed product types.

Results: Forty-five HTA agencies responded to at least one of the surveys. In 2010, twenty-one agencies reported publishing over seventy named HTA products. Core domains associated with full HTA reports were reported by a third of agencies but were labeled differently, so products were classified according to product type (n = 17). Agencies producing short, tailored products increased between 2010 and 2013, with the publication of rapid reviews doubling from 33 percent to 66 percent. In 2013, half of the agencies adapted their common HTA products from documents produced by other agencies. A consensus (>70 percent) was achieved on definitions for HTA reports, mini-HTAs, and rapid reviews.

Conclusions: The product label for an HTA is not always indicative of its content. Terminology has, therefore, been agreed to make explicit the trade-off between rigor and timeliness in three common HTA product types. An INAHTA Product Type (IPT) Mark has been created to identify each of these. It is hoped this will further facilitate HTA adaptation between agencies and reduce duplication of effort.

Accelerated access to innovative medicines for patients in need

There is broad agreement among health-care stakeholders that more must be done to ensure that patients have timely access to new and innovative medicines. Assuming that industry will continue to develop such medicines at a sustainable rate, regulators and payers become the gatekeepers. Regulators, starting in the late 1980s/early 1990s, and, more recently, payers have implemented a variety of early-access pathways or initiatives, and this practice is continuing even today. This article describes the specific approaches that have been taken in four economically developed regions, reviews their success rates, and suggests possible new directions.

Uso da avaliação de tecnologias em saúde em processos de análise para incorporação de tecnologias no Sistema Único de Saúde no Ministério da Saúde

As políticas científicas com objetivos de produção de conhecimentos em saúde foram ampliadas nas últimas décadas. No Brasil, o incentivo à pesquisa, desenvolvimento e inovação em saúde constou na Lei Orgânica da Saúde desde 1990, e políticas científicas e tecnológicas para a área da saúde foram propostas a partir de 1994, incluindo políticas de desenvolvimento da avaliação de tecnologias em saúde (ATS), como em outros países. Coloca-se agora a avaliação dos impactos da ATS sobre a gestão e incorporação de tecnologias nos sistemas de saúde. Como estudo de caso para o uso da ATS em processos de incorporação de tecnologias no Brasil foi analisada a participação do Departamento de Ciência e Tecnologia (DECIT) do, Ministério da Saúde, na Comissão de Incorporação de Tecnologias do Ministério da Saúde (CITEC), no período 2008-2010. Foram utilizados 103 estudos na CITEC, 80% produzidos pelo DECIT, a maioria sínteses com base na literatura existente, e 80% das tecnologias tinham propósito terapêutico, com tendência de crescimento da produção no período. Está em curso um processo de aprendizado tanto metodológico quanto político para o uso da ATS, mas há necessidade de aprofundar a análise do seu impacto sobre o SUS.

SIMILARITIES AND DIFFERENCES BETWEEN FIVE EUROPEAN DRUG REIMBURSEMENT SYSTEMS

Objectives: The aim of our study is to compare five European drug reimbursement systems, describe similarities and differences, and obtain insight into their strengths and weaknesses and formulate policy recommendations.

Methods: We used the analytical Hutton Framework to assess in detail drug reimbursement systems in Austria, Belgium, France, the Netherlands, and Sweden. We investigated policy documents, explored literature, and conducted fifty-seven interviews with relevant stakeholders.

Results: All systems aim to balance three main objectives: system sustainability, equity and quality of care. System impact, however, is mainly assessed by drug expenditure. A national reimbursement agency evaluates reimbursement requests on a case-by-case basis. The minister has discretionary power to alter the reimbursement advice in Belgium, France, and the Netherlands. All systems make efforts to increase transparency in the decision-making process but none uses formal hierarchical reimbursement criteria nor applies a cost-effectiveness threshold value. Policies to deal with uncertainty vary: financial risk-sharing by price/volume contracts (France, Belgium) versus coverage with evidence development (Sweden, the Netherlands). Although case-by-case revisions are embedded in some systems for specific groups of drugs, systematic (group) revisions are limited.

Conclusions: As shared strengths, all systems have clear objectives reflected in reimbursement criteria and all are prepared to pay for drugs with sufficient added value. However, all systems could improve the transparency of the decision-making process; especially appraisal lacks transparency. Systems could increase the use of (systematic) revisions and could make better use of HTA (among others cost-effectiveness) to obtain value for money and ensure system sustainability.

Valorising and creating access to innovative medicines in the European union

This Perspective describes (a) the current situation, (b) challenges and initiatives, (c) and formulates recommendations to valorize and create access to innovative medicines in the EU. We are currently still far away from optimal assessment of value for money in the EU. On the one hand, valorizing innovative medicines involves a local appraisal by health technology assessment (HTA) bodies and competent authorities about the value for money, the budget impact, and the local medical need that can be filled with new medicines. Therefore, local priorities and national health care policy environments should be reflected in the processes and criteria used for assessing value for money and ultimately for reimbursement decisions. On the other hand, a pan-European assessment of both relative effectiveness and medical need (including general ethical and social considerations) should be envisaged in order to feed part of the data needed for the local decisions in an efficient way. This could be the task of the European Medicines Agency, HTA bodies, and competent authorities together.

Health technology assessment: Research trends and future priorities in Europe

Objective:

To provide an overview of health services research related to health technology assessment (HTA) and to identify research priorities from a European perspective.

Methods:

Several methods were used: systematic review of articles indexed with the MeSH term ‘technology assessment’ in PubMed from February 1999-2009; online survey among experts; and conference workshop discussions.

Results:

Research activity in HTA varies considerably across Europe. The research was categorised into six areas: (1) the breadth of analysis in HTA (such as economic, organizational and social aspects); (2) HTA products developed to meet the needs of policy-makers (such as horizon scanning, mini-HTA, and core HTA); (3) handling life-cycle perspectives in relation to technologies; (4) topics that challenge existing methods and for which HTA should be developed to address the themes more comprehensively (such as public health interventions and organizational interventions); (5) development of HTA capacity and programmes; and (6) links between policy and HTA. An online survey showed that the three areas that were given priority were the relationship between HTA and policy-making (71%), the impact of HTA (62%) and incorporating patient aspects in HTA (50%). Policy-makers highlighted HTA and innovation processes as their main research priority (42%). Areas that the systematic review identified as future priorities include issues within the six existing research areas such as disinvestment, developing evidence for new technologies, assessing the wider effects of technology use, and determining how HTA affects decision-making. In addition, relative effectiveness and individualized treatments are areas of growing interest.

Conclusions:

The research priorities identified are important for obtaining high quality and cost-effective health care in Europe. Managing the introduction, use and phasing out of technologies challenges health services throughout Europe, and these processes need to be improved to successfully manage future more general challenges. An ageing population and a diminishing workforce both require strong efforts to ensure effective and well-organized use of human resources and technologies. Furthermore, Europe needs to focus on innovation. This is closely linked to use of technologies and calls for future research.

Involving stakeholders and developing a policy for stakeholder involvement in the European network for health technology assessment, EUnetHTA

OBJECTIVES

This article explains how the issue of stakeholder involvement was addressed in the European network for Health Technology Assessment (EUnetHTA) Project and describes the structures of future stakeholder involvement in the EUnetHTA Collaboration.

METHODS

Initiatives led to a dialogue with stakeholders and exchanging views and expectations on health technology assessment (HTA) processes and the future development of EUnetHTA. The methods of involving different stakeholder groups in EUnetHTA included general information to stakeholders about EUnetHTA, targeted information on a Web site, analysis of stakeholder opinions on HTA and EUnetHTA, and development of a draft stakeholder policy.

RESULTS

First steps were taken to organize processes to consolidate the legitimacy of EUnetHTA and its products and encourage the representation of interests, thus contributing to promoting the utilization of HTA in national/regional policy making. A stakeholder Web site, analyses of stakeholder opinions on HTA and EUnetHTA in a discussion topic catalog, and a draft stakeholder policy resulted from the work.

CONCLUSIONS

Stakeholder involvement in EUnetHTA is necessary to ensure the legitimacy and prospects for utilization of EUnetHTA and its products. The described activities and results create the foundation for a continued dialogue with, and involvement of, stakeholders. The EUnetHTA stakeholder meeting can be considered as a successful experience of dialogue between EUnetHTA and stakeholders, which should be continued. Our experience shows the challenge of obtaining balanced stakeholder representation across the identified stakeholder groups. Continued attention should be given to achieving balanced stakeholder representation.

The health technology assessment adaptation toolkit: description and use

OBJECTIVES

Adapting health technology assessment (HTA) reports for different contexts could reduce the need for multiple reports on the same health technology with resultant saving of time and resources. This article describes an instrument, the adaptation toolkit, which has been developed to aid in the process of adaptation of HTA reports.

METHODS

The toolkit was developed by a partnership of HTA agencies and networks from across Europe. The role of the toolkit is to guide the user through the process of selecting possible relevant material from these report(s), assessing the relevance, reliability, and transferability of the material, and adapting it for the desired context.

RESULTS

The adaptation toolkit has been developed, it comprises a collection of resources that help the user assess whether data and information in existing HTA reports should and could be adapted for their own setting. The toolkit contains two sections: a preliminary speedy sifting section and the main toolkit. The main toolkit includes five domains: (i) technology use and development, (ii) safety, (iii) effectiveness (including efficacy), (iv) economic evaluation, and (v) organizational aspects. Legal, ethical, and social aspects are beyond the scope of the toolkit. The toolkit is designed for the adaptation of evidence synthesis rather than primary research.

CONCLUSIONS

The completed current version of the toolkit contains checklists and resources to aid in the adaptation of HTA reports. This collection of resources is available for use by all HTA agencies and can be accessed at: http://www.eunethta.net/upload/WP5/EUnetHTA_HTA_Adaptation_Toolkit_October08.pdf..

Internal evaluation of the European network for Health Technology Assessment project

Objectives: The internal evaluation studied the development of the European network for Health Technology Assessment (EUnetHTA) Project in achieving the general objective of establishing an effective and a sustainable network of health technology assessment (HTA) in Europe.

Methods: The Work Package 3 group was dedicated to this task and performed the work. Information on activities during the project was collected from three sources. First, three yearly cross-sectional studies surveyed the participants’ opinions. Responses were by individuals or by institutions. The last round included surveys to the Steering Committee, the Stakeholder Forum, and the Secretariat. Second, the Work Package Lead Partners were interviewed bi-annually, five times in total, to update the information on the Project's progress. Third, additional information was sought in available documents.

Results: The organizational structure remained stable. The Project succeeded in developing tools aimed at providing common methodology with intent to establish a standard of conducting and reporting HTA and to facilitate greater collaboration among agencies. The participants/agencies expressed their belief in a network and in maintaining local/national autonomy. The Work Package Leaders expressed a strong belief in the solid base of the Project for a future network on which to build, but were aware of the need for funding and governmental support.

Conclusions: Participants and Work Package Leaders have expressed support for a future network that will improve national and international collaboration in HTA based on the experience from the EUnetHTA project.

Web-based toolkit to facilitate European collaboration on evidence generation on promising health technologies

Background: Several countries have developed policy frameworks allowing timely access to promising health technologies on the condition that additional evidence is generated. However, an important barrier to evidence generation is the lack of structured collaboration among health technology assessment (HTA) agencies.

Objectives: One of the aims of Work Package 7 (WP7) of the European network for Health Technology Assessment (EUnetHTA) Project was to determine the types of structured collaboration that could facilitate evidence generation and to develop a Web-based toolkit to support such collaboration.

Methods: Collaboration modalities were defined by all WP7 Partners. Initial emphasis was on information sharing. Standardized forms for information sharing were developed and tested. An information technology development phase followed with the creation of the Web-based toolkit (Web site).

Results: Three levels of collaboration were agreed on: (i) sharing information, (ii) coordinated action, and (iii) joint action. The Web site allows access to structured and standardized forms for requesting information, posting information in response to a request, and posting information spontaneously. An online database contains all of the information requested or posted. Pilot tests on twenty-one promising technologies were satisfactory.

Conclusions: This new Web site for sharing information on evidence generation should help countries reach robust decisions on the timely adoption of promising health technologies. It will only become fully operational if EUnetHTA Partners supply relevant, accurate, and updated information, and regularly use the Web site.

Toward a new information infrastructure in health technology assessment: Communication, design, process, and results

Objectives: The aim of this study was to facilitate effective internal and external communication of an international network and to explore how to support communication and work processes in health technology assessment (HTA).

Structure and Methods: European network for Health Technology Assessment (EUnetHTA) connected sixty-four HTA Partner organizations from thirty-three countries. User needs in the different steps of the HTA process were the starting point for developing an information system. A step-wise, interdisciplinary, creative approach was used in developing practical tools.

Results: An Information Platform facilitated the exchange of scientific information between Partners and with external target groups. More than 200 virtual meetings were set up during the project using an e-meeting tool. A Clearinghouse prototype was developed with the intent to offering a single point of access to HTA relevant information. This evolved into a next step not planned from the outset: Developing a running HTA Information System including several Web-based tools to support communication and daily HTA processes. A communication strategy guided the communication effort, focusing on practical tools, creating added value, involving stakeholders, and avoiding duplication of effort.

Conclusions: Modern technology enables a new information infrastructure for HTA. The potential of information and communication technology was used as a strategic tool. Several target groups were represented among the Partners, which supported collaboration and made it easier to identify user needs. A distinctive visual identity made it easier to gain and maintain visibility on a limited budget.

Information service on new and emerging health technologies: Identification and prioritization processes for a European Union–wide newsletter

Objectives: EUnetHTA WP 7 (Strand B) aimed to promote sharing information on new and emerging technologies. The task was to develop a prototype of a newsletter and pilot the processes of production.

Methods: The EuroScan database served as information source on pertinent technologies. To prioritize, a set of criteria for scoring the potential impact and for selecting the technologies for articles was applied and a pilot newsletter was produced.

Results: Being objective and transparent about the content of a newsletter required a method for prioritizing health technologies. Using significance criteria, members of the prioritization panel selected twelve technologies for articles of different length and depth. Potential recipients, surveyed on relevance, content, timeliness, and readability responded mostly positive, but requested more information on cost effectiveness and criticized timeliness.

Conclusions: Dissemination of an EU-wide newsletter would be feasible, but time-consuming. Although a newsletter appears to fulfill a need for information on emerging and new health technologies, it is not considered the right tool to avoid duplication of effort in the present international constellation of horizon scanning for new health technologies. Other options will be pursued as part of future collaborative actions, for example, a core set of early awareness information, or an on-demand electronic information system.

Enhancing understanding: The development of a glossary of health technology assessment adaptation terms

Background: The way people use health technology assessment (HTA) terms varies considerably across Europe. Such variation can lead to misunderstandings when reading HTA reports from different contexts. This work is one of the outputs of the EUnetHTA Project and was undertaken between 2006 and 2008.

Objectives: The aim of this study was to develop a glossary of HTA adaptation terms to help reduce the misunderstandings of terms used in HTA reports from contexts other than the reader's own.

Methods: Several HTA glossaries were examined to identify ways in which an additional glossary could offer readers something new and to identify adaptation terms for inclusion. Twenty-eight European HTA organizations provided terms for the glossary and drafted descriptions and examples of how each specific term was used in their particular setting. The organizations then commented on the descriptions provided by the other groups and worked together to draft a single description for certain terms.

Results: A glossary of HTA adaptation terms was developed. It provides a comprehensive range of descriptions, examples, and comments for forty-two potentially confusing HTA terms related to adaptation.

Conclusions: This glossary will be a valuable resource for European HTA agencies when reading HTA reports produced in different contexts and for adapting HTA reports produced in other countries. The glossary will help improve understanding and help facilitate the adaptation process.