Are Key Principles for improved health technology assessment supported and used by health technology assessment organizations?

National health technology assessment in Turkiye after a decade: are key principles followed?

BACKGROUND

Health technology assessment (HTA) is growing in low- and middle-income countries (LMICs) to ensure optimal use of limited resources. However, the impact of HTAs on decision making in LMICs has been limited. The study aimed to provide an overview of Turkiye's progress since establishing the first HTA agency in 2012.

METHODS

The web sites of three national HTA agencies in Turkiye were searched for HTA guidelines and national HTA reports. The HTA guidelines were assessed by two researchers independently against the key principles of HTA developed by Drummond et al., and the HTA reports against the national guidelines.

RESULTS

The study included one HTA guideline and eight national HTA reports. The guideline included very limited technical guidance. Compliance with the principles was poor to moderate, and significant methodological limitations were identified. The reports were inconsistent regarding the scope and the HTA assessment criteria. The link between HTA findings, HTA decision making, and health policies were not clear.

DISCUSSION

The inconsistencies between the reports and the methodological limitations demonstrate the need for national HTA guidelines. Improving the characteristics of the HTA might impact implementation. Among the key issues is transparency regarding priority setting, the HTA process, and decision making.

CONCLUSION

Establishing and adopting national HTA guidelines at international standards is needed. Involving external scientific committees and health economists in the HTA processes might help ensure that the key principles of HTA are followed. The study findings might be helpful for countries that are developing their HTA systems.

Values, principles, strategies, and frameworks underlying patient and public involvement in health technology assessment and guideline development: a scoping review

The importance of patient and public involvement (PPI) is recognized by agencies involved in health technology assessment (HTA) and guideline development. However, a comprehensive overview of the underlying PPI principles, values, strategies, and frameworks is lacking. This scoping review aimed to summarize the available evidence on principles, values, frameworks, and strategies underpinning PPI carried out by agencies involved in HTA and guideline development. A total of twelve records were included, of which four referred to guidelines and eight to HTA. Overall, this review demonstrated a lack of consistency in the definition and application of the concepts of values and principles to PPI in the context of guideline development and HTA. There was significant overlap between values and principles, with some broad themes emerging, such as representation, transparency, relevance, equity, fairness, and reconciling different types of knowledge. Frameworks were typically based on the stages of guideline development or HTA, despite heterogeneity in how stages were labeled and described. Strategies were also mapped to the stages of guideline development and HTA and varied substantially depending on the context and setting. Both strategies and frameworks demonstrated patients and the public can be involved, albeit to a variable extent, throughout the stages of guideline development and HTA. However, frameworks often failed to explicitly link the values and principles with the HTA and guideline development stages through actionable PPI strategies. Further research is warranted to better understand the values, principles, and frameworks underpinning PPI in guideline development and HTA.

Topic selection process in health technology assessment agencies around the world: a systematic review

Objective

The purpose of this study was to systematically review the process for topic selection by health technology assessment (HTA) agencies around the world to provide the knowledge base for the improvement of topic selection frameworks in HTA agencies.

Methods

A systematic search was conducted in PubMed and EMBASE to identify papers up to February 2019. Gray literature was identified by screening the Web sites of HTA agencies on the nonprofit member list of the International Network of Agencies for Health Technology Assessment (INAHTA). Data were extracted for each HTA agency and synthesized, with issues including general contextual information about each agency and the process of topic selection.

Results

Out of forty-nine nonprofit members of INAHTA, a total of seventeen HTA agencies with a framework for topic selection were identified from twenty-two included papers/documents. Multiple criteria were used for topic selection in all frameworks and agencies undertook multiple steps, which could include the specification of criteria for topic selection, identification of topics, short listing of potential topics, scoping of potential topics, scoring and ranking of potential topics, and deliberation and decision on final topics for HTA. Shortcomings were found in relation to methods of scoring and ranking as well as lack of monitoring and the evaluation of the process.

Conclusions

Our study provides insights into the current practice of topic selection in HTA agencies. Multiple criteria decision analysis methodology appears highly relevant to these processes. A consensus approach for the development of methods of topic selection would be valuable for the HTA community.

Benchmarking health technology assessment agencies-methodological challenges and recommendations

OBJECTIVES

The objectives of the study were to establish a benchmarking tool to collect metrics to enable increased clarity regarding the differences and similarities across health technology assessment (HTA) agencies, to assess performance within and across HTA agencies, identify areas in the HTA processes in which time is spent and to enable ongoing performance improvement.

METHODS

Common steps and milestones in the HTA process were identified for meaningful benchmarking among agencies. A benchmarking tool consisting of eighty-six questions providing information on HTA agency organizational aspects and information on individual new medicine review timelines and outcomes was developed with the input of HTA agencies and validated in a pilot study. Data on 109 HTA reviews from five HTA agencies were analyzed to demonstrate the utility of this tool.

RESULTS

This study developed an HTA benchmarking methodology, comparative metrics showed considerable differences among the median timelines from assessment and appraisal to final HTA recommendation for the five agencies included in this analysis; these results were interpreted in conjunction with agency characteristics.

CONCLUSIONS

It is feasible to find consensus among HTA agencies regarding the common milestones of the review process to map jurisdiction-specific processes against agreed metrics. Data on characteristics of agencies such as their scope and remit enabled results to be interpreted in the appropriate local context. This benchmarking tool has promising potential utility to improve the transparency of the review process and to facilitate both quality assurance and performance improvement in HTA agencies.

Health technology assessment of biosimilars worldwide: a scoping review

BACKGROUND

Health technology assessment (HTA) should provide an assessment of a technology's effects on health and of the related social, economic, organisational and ethical issues. HTA reports on biosimilars can specifically assess their immunogenicity, their extrapolation to one or more conditions, and the risks of interchangeability and substitution. We aimed to complete a scoping review within the context of HTA organisations to synthesise HTA reports on biosimilars and to map the extension, scope and methodological practices.

MAIN BODY

A scoping review methodology was applied. The sources for biosimilars HTA reports were database searches and grey literature from HTA organisation websites up to June 2019. HTA reports of biosimilars were classified as full HTA, mini-HTA or rapid reviews. Data were extracted and recorded on a calibrated predefined data form. We identified 70 HTA reports of biosimilars of 16 biologic products (65.71% in 2015-2018) produced by 13 HTA organisations from 10 countries; 2 full HTAs, 4 mini-HTAs and 64 rapid reviews met the inclusion criteria. Almost all the rapid reviews gave no information regarding any evidence synthesis method and approximately half of the rapid reviews did not appraise the risk of bias of primary studies or the overall quality of evidence. All full-HTAs and mini-HTAs addressed organisational, ethical, social and legal considerations, while these factors were assessed in less than half of the rapid reviews. The immunogenicity and extrapolation of one or more conditions were often considered. The majority of full-HTAs and mini-HTAs contained an assessment of switching and a discussion of an educational approach about biosimilars. No HTA report rejected the adoption/reimbursement of the biosimilar assessed.

CONCLUSION

HTA of biosimilars are emerging in the context of HTA organisations and those that exist often duplicate reports of the same biosimilar. Most HTA reports of biosimilars do not conduct a systematic literature review or consider economic issues. No report has rejected the adoption/reimbursement of biosimilars. There is a need to standardise the minimum criteria for the development of HTA on biosimilars to ensure a better understanding and better decision-making.

Evaluating the effects of a risk-adapted screening program for familial colorectal cancer in individuals between 25 and 50 years of age: study protocol for the prospective population-based intervention study FARKOR

BACKGROUND

Colorectal cancer (CRC) is the second most common malignant disease and the second most common cause of cancer death in Germany. Official CRC screening starts at age 50. As there is evidence that individuals with a family history of CRC have an increased risk of developing CRC before age 50, there are recommendations to start screening for this group earlier. This study aims to evaluate the clinical and economic effects of a risk-adapted screening program for CRC in individuals between 25 and 50 years of age with potentially increased familial CRC risk.

METHODS

FARKOR (Familiäres Risiko für das Kolorektale Karzinom) is a population-based prospective intervention study. All members of cooperating statutory health insurance companies between 25 and 50 years of age living in a model region in Germany (federal state of Bavaria, 3.5 million inhabitants in this age group) can participate in the program between October 2018 and March 2020. Recruitment takes place through physicians and through a public campaign. Additionally, insurances contact recently diagnosed CRC patients in order to encourage their relatives to participate in the program. Physicians assess a participant's familial history of CRC using a short questionnaire. All participants with a family history of CRC are invited to a shared decision making process to decide on further screening options consisting of either undergoing an immunological test for fecal occult blood or colonoscopy. Comprehensive data collection based on self-reported lifestyle information, medical documentation and health administrative databases accompanies the screening program. Longterm benefits, harms and the cost-effectiveness of the risk-adapted CRC screening program will be assessed by decision analytic modeling.

DISCUSSION

The data collected in this study will add important pieces of information that are still missing in the evaluation of the effects and the cost-effectiveness of a risk-adapted CRC screening strategy for individuals under 50 years of age.

TRIAL REGISTRATION

German Clinical Trials Register, DRKS-IDDRKS00015097.

Use of Evidence-Informed Deliberative Processes by Health Technology Assessment Agencies Around the Globe

Background: Evidence-informed deliberative processes (EDPs) were recently introduced to guide health technology assessment (HTA) agencies to improve their processes towards more legitimate decision-making. The EDP framework provides guidance that covers the HTA process, ie, contextual factors, installation of an appraisal committee, selecting health technologies and criteria, assessment, appraisal, and communication and appeal. The aims of this study were to identify the level of use of EDPs by HTA agencies, identify their needs for guidance, and to learn about best practices.

Methods: A questionnaire for an online survey was developed based on the EDP framework, consisting of elements that reflect each part of the framework. The survey was sent to members of the International Network of Agencies for Health Technology Assessment (INAHTA). Two weeks following the invitation, a reminder was sent. The data collection took place between September-December 2018.

Results: Contact persons from 27 member agencies filled out the survey (response rate: 54%), of which 25 completed all questions. We found that contextual factors to support HTA development and the critical elements regarding conducting and reporting on HTA are overall in place. Respondents indicated that guidance was needed for specific elements related to selecting technologies and criteria, appraisal, and communication and appeal. With regard to best practices, the Canadian Agency for Drugs and Technologies and the National Institute for Health and Care Excellence (NICE, UK) were most often mentioned.

Conclusion: This is the first survey among HTA agencies regarding the use of EDPs and provides useful information for further developing a practical guide for HTA agencies around the globe. The results could support HTA agencies in improving their processes towards more legitimate decision-making, as they could serve as a baseline measurement for future monitoring and evaluation.

Advanced therapy medicinal products and health technology assessment principles and practices for value-based and sustainable healthcare

BACKGROUND

Advanced therapy medicinal products (ATMPs) are beginning to reach European markets, and questions are being asked about their value for patients and how healthcare systems should pay for them.

OBJECTIVES

To identify and discuss potential challenges of ATMPs in view of current health technology assessment (HTA) methodology-specifically economic evaluation methods-in Europe as it relates to ATMPs, and to suggest potential solutions to these challenges.

METHODS

An Expert Panel reviewed current HTA principles and practices in relation to the specific characteristics of ATMPs.

RESULTS

Three key topics were identified and prioritised for discussion-uncertainty, discounting, and health outcomes and value. The panel discussed that evidence challenges linked to increased uncertainty may be mitigated by collection of follow-on data, use of value of information analysis, and/or outcomes-based contracts. For discount rates, an international, multi-disciplinary forum should be established to consider the economic, social and ethical implications of the choice of rate. Finally, consideration of the feasibility of assessing the value of ATMPs beyond health gain may also be key for decision-making.

CONCLUSIONS

ATMPs face a challenge in demonstrating their value within current HTA frameworks. Consideration of current HTA principles and practices with regards to the specific characteristics of ATMPs and continued dialogue will be key to ensuring appropriate market access.

CLASSIFICATION CODE

I.

Health Technology Optimization Analysis: Conceptual Approach and Illustrative Application

We present a conceptual approach to determine the optimal solution to delivering a health technology, consistent with the objective of maximizing patient outcomes subject to resources available to a publicly funded health system. The article addresses two key policy questions: 1) adding system values through appropriate planning of health services delivery and 2) considering the tradeoff between patient outcomes and costs to the health system through appropriate use of health technologies for conditions with time-dependent treatment outcomes. We develop a health technology optimization framework that considers geographical variation and searches for the best delivery method through a pairwise comparison of all possible strategies, factoring in controlled variables including disease epidemiology, time or distance to hospitals, available medical services, treatment eligibility, treatment efficacy, and costs. Taking variations of these factors into account would help support a more efficient allocation of health resources. Drawing identified strategies together then creates a map of optimal strategies. We apply the proposed method to a policy-relevant health technology assessment of endovascular therapy (EVT) for treating acute ischemic stroke. The best strategy for providing EVT relies on the geographical location of stroke onset and the decision maker’s preference for either patient outcomes or economic efficiency. The proposed method produced an optimization map showing the optimal strategy for EVT delivery, which maximizes patient outcomes while minimizing health system costs. In the illustrative case study, there were no tradeoffs between health outcomes and costs, meaning that the delivery strategies that were clinically optimal for patients were also the most cost-effective. In conclusion, the health technology optimization approach is a useful tool for informing implementation decisions and coordinating the delivery of complex health services such as EVT.

HEALTH TECHNOLOGY ASSESSMENT FOR DECISION MAKING IN LATIN AMERICA: GOOD PRACTICE PRINCIPLES

OBJECTIVES

The aim of this study was to identify good practice principles for health technology assessment (HTA) that are the most relevant and of highest priority for application in Latin America and to identify potential barriers to their implementation in the region.

METHODS

HTA good practice principles proposed at the international level were identified and then explored during a deliberative process in a forum of assessors, funders, and product manufacturers.

RESULTS

Forty-two representatives from ten Latin American countries participated. Good practice principles proposed at the international level were considered valid and potentially relevant to Latin America. Five principles were identified as priority and with the greatest potential to be strengthened at this time: transparency in the production of HTA, involvement of relevant stakeholders in the HTA process, mechanisms to appeal decisions, clear priority-setting processes in HTA, and a clear link between HTA and decision making. The main challenge identified was to find a balance between the application of these principles and the available resources in a way that would not detract from the production of reports and adaptation to the needs of decision makers.

CONCLUSIONS

The main recommendation was to progress gradually in strengthening HTA and its link to decision making by developing appropriate processes for each country, without trying to impose, in the short-term, standards taken from examples at the international level without adequate adaptation of these to local contexts.

Towards Integrated Health Technology Assessment for Improving Decision Making in Selected Countries

OBJECTIVES

To assess the level of comprehensiveness of health technology assessment (HTA) practices around the globe and to formulate recommendations for enhancing legitimacy and fairness of related decision-making processes.

METHODS

To identify best practices, we developed an evaluation framework consisting of 13 criteria on the basis of the INTEGRATE-HTA model (integrative perspective on assessing health technologies) and the Accountability for Reasonableness framework (deliberative appraisal process). We examined different HTA systems in middle-income countries (Argentina, Brazil, and Thailand) and high-income countries (Australia, Canada, England, France, Germany, Scotland, and South Korea). For this purpose, desk research and structured interviews with relevant key stakeholders (N = 32) in the selected countries were conducted.

RESULTS

HTA systems in Canada, England, and Scotland appear relatively well aligned with our framework, followed by Australia, Germany, and France. Argentina and South Korea are at an early stage, whereas Brazil and Thailand are at an intermediate level. Both desk research and interviews revealed that scoping is often not part of the HTA process. In contrast, providing evidence reports for assessment is well established. Indirect and unintended outcomes are increasingly considered, but there is room for improvement. Monitoring and evaluation of the HTA process is not well established across countries. Finally, adopting transparent and robust processes, including stakeholder consultation, takes time.

CONCLUSIONS

This study presents a framework for assessing the level of comprehensiveness of the HTA process in a country. On the basis of applying the framework, we formulate recommendations on how the HTA community can move toward a more integrated decision-making process using HTA.

QUALITY OF HEALTH TECHNOLOGY ASSESSMENT REPORTS PREPARED FOR THE MEDICAL SERVICES ADVISORY COMMITTEE

Objectives:The Medical Services Advisory Committee (MSAC) makes recommendations to the Australian Government for funding health technologies under the Medicare Benefits Schedule (MBS). Differences in public, clinical, commercial, and political opinions on health expenditure emphasize the importance of defensible funding decisions. We aimed to evaluate the quality of health technology assessment (HTA) reports over time and among health technologies assessed for MSAC.

Main Outcome Measures:A cohort study was performed of HTA reports prepared for MSAC between 1998 and 2013. We measured the quality of HTA reports using reporting guidelines proposed by the European Collaboration for Assessment of Health Interventions. Individual component scores across eleven domains were calculated, and summed for an overall aggregate score. We used linear regression to investigate any change in quality over time and among the types of technologies assessed.

Results:We included 110 HTA reports. The safety (80 percent), effectiveness (84 percent), economic (74 percent), and organizational (99 percent) domains were better reported than the psychological, social, and ethical considerations (34 percent). The basic (75 percent), methodological (62 percent), background (82 percent), contextual (46 percent), status quo (54 percent), and technical information (66 percent) that framed each assessment were inconsistently reported. On average, overall quality scores increased by 2 percent (p< 0.001) per year, from approximately 60 percent to 80 percent over the 15-year period, with no significant difference among surgical, diagnostic or other nonpharmaceutical health technologies (p= 0.22).

Conclusions:HTA reports prepared for MSAC are a key tool in allocating scarce health resources. The overall quality of these reports has improved, but the reporting of specific domains and subthemes therein could be better addressed.

GUIDING PRINCIPLES FOR GOOD PRACTICES IN HOSPITAL-BASED HEALTH TECHNOLOGY ASSESSMENT UNITS

Objectives: Health technology assessment (HTA) carried out for policy decision making has well-established principles unlike hospital-based HTA (HB-HTA), which differs from the former in the context characteristics and ways of operation. This study proposes principles for good practices in HB-HTA units.

Methods: A framework for good practice criteria was built inspired by the EFQM excellence business model and information from six literature reviews, 107 face-to-face interviews, forty case studies, large-scale survey, focus group, Delphi survey, as well as local and international validation. In total, 385 people from twenty countries have participated in defining the principles for good practices in HB-HTA units.

Results: Fifteen guiding principles for good practices in HB-HTA units are grouped in four dimensions. Dimension 1 deals with principles of the assessment process aimed at providing contextualized information for hospital decision makers. Dimension 2 describes leadership, strategy and partnerships of HB-HTA units which govern and facilitate the assessment process. Dimension 3 focuses on adequate resources that ensure the operation of HB-HTA units. Dimension 4 deals with measuring the short- and long-term impact of the overall performance of HB-HTA units. Finally, nine core guiding principles were selected as essential requirements for HB-HTA units based on the expertise of the HB-HTA units participating in the project.

Conclusions: Guiding principles for good practices set up a benchmark for HB-HTA because they represent the ideal performance of HB-HTA units; nevertheless, when performing HTA at hospital level, context also matters; therefore, they should be adapted to ensure their applicability in the local context.

CHALLENGES FACED IN TRANSFERRING ECONOMIC EVALUATIONS TO MIDDLE INCOME COUNTRIES

BACKGROUND

Decision makers in middle-income countries are using economic evaluations (EEs) in pricing and reimbursement decisions for pharmaceuticals. However, whilst many of these jurisdictions have local submission guidelines and local expertise, the studies themselves often use economic models developed elsewhere and elements of data from countries other than the jurisdiction concerned. The objectives of this study were to describe the current situation and to assess the challenges faced by decision makers in transferring data and analyses from other jurisdictions.

METHODS

Experienced health service researchers in each region conducted an interview survey of representatives of decision making bodies from jurisdictions in Asia, Central and Eastern Europe, and Latin America that had at least 1 year's experience of using EEs.

RESULTS

Representatives of the relevant organizations in twelve countries were interviewed. All twelve jurisdictions had developed official guidelines for the conduct of EEs. All but one of the organizations evaluated studies submitted to them, but 9 also conducted studies and 7 commissioned them. Nine of the organizations stated that, in evaluating EEs submitted to them, they had consulted a study performed in a different jurisdiction. Data on relevant treatment effect was generally considered more transferable than those on prices/unit costs. Views on the transferability of epidemiological data, data on resource use and health state preference values were more mixed. Eight of the respondents stated that analyses submitted to them had used models developed in other jurisdictions. Four of the organizations had a policy requiring models to be adapted to reflect local circumstances. The main obstacles to transferring EEs were the different patterns of care or wealth of the developed countries from which most economic evaluations originate.

CONCLUSIONS

In middle-income countries it is commonplace to deal with the issue of transferring analyses or data from other jurisdictions. Decision makers in these countries face several challenges, mainly due to differences in current standard of care, practice patterns, or gross domestic product between the developed countries where the majority of the studies are conducted and their own jurisdiction.

Evaluating the quality and use of economic data in decisions about essential medicines

Objective

To evaluate the quality of economic data provided in applications to the World Health Organization (WHO) Model List of Essential Medicines and to evaluate the role of these data in decision-making by the expert committee that considers the applications.

Methods

We analysed applications submitted to the WHO Expert Committee on the Selection and Use of Essential Medicines between 2002 and 2013. The completeness of data on the price and cost–effectiveness of medicines was extracted from application documents and coded using a four-point scale. We recorded whether or not the expert committee discussed economic information and the outcomes of each application. Associations between the completeness of economic data and application outcomes were assessed using χ² tests.

Findings

The expert committee received 134 applications. Only eight applications (6%) included complete price data and economic evaluation data. Many applicants omitted or misinterpreted the economic evaluation section of the application form. Despite the lack of economic data, all applications were reviewed by the committee. There was no significant association between the completeness of economic information and application outcomes. The expert committee tried to address information gaps in applications by further review and analysis of data related to the application.

Conclusion

The World Health Organization should revise the instructions to applicants on economic data requirements; develop new mechanisms to assist applicants in completing the application process; and define methods for the use of economic data in decision-making.

Evolution of health technology assessment: best practices of the pan-Canadian Oncology Drug Review

Background

In 2007, Canada chose to develop a separate and distinct path for oncology drug health technology assessment (HTA). In 2013, the decision was made to transfer the pan-Canadian Oncology Drug Review (pCODR) to the Canadian Agency for Drugs and Technologies in Health (CADTH), to align the pCODR and CADTH Common Drug Review processes while building on the best practices of both. The objective of this research was to conduct an examination of the best practices established by the pCODR.

Methods

A qualitative research approach was taken to assess the policies, processes, and practices of the pCODR, based on internationally accepted best practice “principles” in HTA, with a particular focus on stakeholder engagement. Publicly available information regarding the approach of the pCODR was used to gauge the agency’s performance against these principles. In addition, stakeholder observations and real-world experiences were gathered through key informant interviews to be inclusive of perspectives from patient advocacy groups, provincial and/or cancer agency decision-makers, community and academic oncologists, industry, expert committee members, and health economists.

Results

This analysis indicated that, through the pCODR, oncology stakeholders have had a voice in and have come to trust the quality and relevance of oncology HTA as a vital tool to ensure the best decisions for Canadians with cancer and their health care system. It could be expected that adoption of the principles and processes of the pCODR would bring a similar level of engagement and trust to other HTA organizations in Canada and elsewhere.

Conclusion

The results of this research led to recommendations for improvement and potential extrapolation of these best practices to other HTA organizations worldwide, along with suggestions for continued evolution of the pCODR in conjunction with its integration into the CADTH. It is clear that the transition of the pCODR to CADTH provides an opportunity for practices initiated by the pCODR to become the standard for these newly amalgamated HTA agencies in Canada.

Conflicts of interest and critiques of the use of systematic reviews in policymaking: an analysis of opinion articles

BACKGROUND

Strong opinions for or against the use of systematic reviews to inform policymaking have been published in the medical literature. The purpose of this paper was to examine whether funding sources and author financial conflicts of interest were associated with whether an opinion article was supportive or critical of the use of systematic reviews for policymaking. We examined the nature of the arguments within each article, the types of disclosures present, and whether these articles are being cited in the academic literature.

METHODS

We searched for articles that expressed opinions about the use of systematic reviews for policymaking. We included articles that presented opinions about the use of systematic reviews for policymaking and categorized each article as supportive or critical of such use. We extracted all arguments regarding the use of systematic reviews from each article and inductively coded each as internal or external validity argument, categorized disclosed funding sources, conflicts of interest, and article types, and systematically searched for undisclosed financial ties. We counted the number of times each article has been cited in the "Web of Science." We report descriptive statistics.

RESULTS

Articles that were critical of the use of systematic reviews (n=25) for policymaking had disclosed or undisclosed industry ties 2.3 times more often than articles that were supportive of the use (n=34). We found that editorials, comments, letters, and perspectives lacked published disclosures nearly twice as often (60% v. 33%) as other types of articles. We also found that editorials, comments, letters, and perspectives were less frequently cited in the academic literature than other article types (median number of citations=5 v. 19).

CONCLUSIONS

It is important to consider whether an article has industry ties when evaluating the strength of the argument for or against the use of systematic reviews for policymaking. We found that journal conflict of interest disclosures are often inadequate, particularly for editorials, comments, letters, and perspectives and that these articles are being cited as evidence in the academic literature. Our results further suggest the need for more consistent and complete disclosure for all article types.

DOES HEALTH TECHNOLOGY ASSESSMENT AFFECT POLICY-MAKING AND CLINICAL PRACTICE IN SWEDEN?

Objectives: The aim of this study was to analyze whether health technology assessment (HTA) reports published by SBU have influenced decisions, guidelines, clinical practice, or research priorities in Sweden.

Methods: All twenty-six SBU reports between 2006 and 2010 were analyzed. For each project, we searched publications and documentation that reflected impact on decisions, guidelines, research or clinical practice. Written documentation, before–after surveys or register-based time series data were used when available. Based on a conceptual model and on the available evidence, we determined whether HTA reports had a high, moderate, or low impact.

Results: HTA reports influenced comprehensive decisions to a high or moderate degree. In the case of fortifying flour with folic acid to a high degree. In ten cases, HTA reports were the primary source of clinical guidelines developed by the National Board of Health and Welfare (NBHW) or professional associations. In the cases of dyspepsia and gastro-esophageal reflux, as well as mild head injury, the HTA reports had a high impact on clinical practice. It was also obvious from this review that research had been initiated as a result of the knowledge gaps identified by HTA reports. In three cases, we had no adequate documentation, suggesting that the impact of the HTA report had been low.

Conclusions: Many interrelated forces change practice, but the cases presented here indicate that HTA reports have had a high impact on clinical guidelines, as well as a moderate or high impact on comprehensive decisions, the initiation of research and changes in clinical practice.

Methods of international health technology assessment agencies for economic evaluations--a comparative analysis

Background

The number of Health Technology Assessment (HTA) agencies increases. One component of HTAs are economic aspects. To incorporate economic aspects commonly economic evaluations are performed. A convergence of recommendations for methods of health economic evaluations between international HTA agencies would facilitate the adaption of results to different settings and avoid unnecessary expense. A first step in this direction is a detailed analysis of existing similarities and differences in recommendations to identify potential for harmonization. The objective is to provide an overview and comparison of the methodological recommendations of international HTA agencies for economic evaluations.

Methods

The webpages of 127 international HTA agencies were searched for guidelines containing recommendations on methods for the preparation of economic evaluations. Additionally, the HTA agencies were requested information on methods for economic evaluations. Recommendations of the included guidelines were extracted in standardized tables according to 13 methodological aspects. All process steps were performed independently by two reviewers.

Results

Finally 25 publications of 14 HTA agencies were included in the analysis. Methods for economic evaluations vary widely. The greatest accordance could be found for the type of analysis and comparator. Cost-utility-analyses or cost-effectiveness-analyses are recommended. The comparator should continuously be usual care. Again the greatest differences were shown in the recommendations on the measurement/sources of effects, discounting and in the analysis of sensitivity. The main difference regarding effects is the focus either on efficacy or effectiveness. Recommended discounting rates range from 1.5% - 5% for effects and 3% - 5% for costs whereby it is mostly recommended to use the same rate for costs and effects. With respect to the analysis of sensitivity the main difference is that oftentimes the probabilistic or deterministic approach is recommended exclusively. Methods for modeling are only described vaguely and mainly with the rational that the “appropriate model” depends on the decision problem. Considering all other aspects a comparison is challenging as recommendations vary regarding detailedness and addressed issues.

Conclusion

There is a considerable unexplainable variance in recommendations. Further effort is needed to harmonize methods for preparing economic evaluations.

Assessing the added value of health technologies: reconciling different perspectives

Providing universal access to innovative, high-cost technologies leads to tensions in today's health care systems. The tension becomes particularly evident in the context of scarce resources, where the risk of taking contentious coverage decisions increases rapidly. To ensure economic sustainability, the payers of health care think that the benefits from the use of the new technologies need to be commensurate with the costs. Therefore, many jurisdictions have programs of health technology assessment, which often results in restrictions of access to care, either through complete refusal to reimburse the technology or its restriction of use to only a subset of the eligible patient population. However, manufacturers feel that they should be adequately rewarded for their innovations and require sufficient funds to invest in further research. Finally, patients perceive these technologies to have added benefits, and so they are concerned when they are denied access. If sustainable access to health care is to be maintained in the future, approaches are needed to reconcile these different perspectives. This article explores the approaches, in both methods and policy, to help bring about this reconciliation. These include rethinking the notion of social value (on the part of payers), aligning manufacturers' research more closely with societal objectives, and increasing patient participation in health technology assessment.

Uso da avaliação de tecnologias em saúde em processos de análise para incorporação de tecnologias no Sistema Único de Saúde no Ministério da Saúde

As políticas científicas com objetivos de produção de conhecimentos em saúde foram ampliadas nas últimas décadas. No Brasil, o incentivo à pesquisa, desenvolvimento e inovação em saúde constou na Lei Orgânica da Saúde desde 1990, e políticas científicas e tecnológicas para a área da saúde foram propostas a partir de 1994, incluindo políticas de desenvolvimento da avaliação de tecnologias em saúde (ATS), como em outros países. Coloca-se agora a avaliação dos impactos da ATS sobre a gestão e incorporação de tecnologias nos sistemas de saúde. Como estudo de caso para o uso da ATS em processos de incorporação de tecnologias no Brasil foi analisada a participação do Departamento de Ciência e Tecnologia (DECIT) do, Ministério da Saúde, na Comissão de Incorporação de Tecnologias do Ministério da Saúde (CITEC), no período 2008-2010. Foram utilizados 103 estudos na CITEC, 80% produzidos pelo DECIT, a maioria sínteses com base na literatura existente, e 80% das tecnologias tinham propósito terapêutico, com tendência de crescimento da produção no período. Está em curso um processo de aprendizado tanto metodológico quanto político para o uso da ATS, mas há necessidade de aprofundar a análise do seu impacto sobre o SUS.

Therapies for advanced cancers pose a special challenge for health technology assessment organizations in many countries

Health technology assessment organizations evaluate medical therapies and technologies to help inform coverage and reimbursement decisions for payers around the globe. Even as they establish strict review processes, these organizations--and the reimbursement authorities that use their assessments--have sometimes handled cancer interventions with special care. We found that some countries have created separate health technology assessment pathways for cancer treatment, while others have eased access to cancer treatments through end-of-life or disease-severity exceptions within health technology assessment policies. In the United States, although no separate evaluation pathways exist for cancer, cancer drugs receive special status by virtue of unique Medicare rules covering off-label indications. Worldwide, we demonstrate that health technology assessment organizations are struggling with cancer's "exceptionalism."

Omalizumab for the treatment of severe persistent allergic asthma in children aged 6-11 years: a NICE single technology appraisal

Following a licence extension to include those aged 6-11 years, the National Institute for Health and Clinical Excellence (NICE) invited the manufacturer of omalizumab (Novartis Pharmaceuticals UK) to submit evidence for the clinical and cost effectiveness of this drug for patients with severe persistent allergic asthma in this age bracket. NICE had previously considered the use of omalizumab in patients aged 12 years and over. The Centre for Reviews and Dissemination (CRD) and the Centre for Health Economics (CHE) at the University of York were commissioned as the Evidence Review Group (ERG) to critically appraise the evidence presented by the manufacturer. This article summarizes that review of the evidence, the deliberations of the NICE Appraisal Committee and the resulting NICE guidance. The ERG critically reviewed the evidence presented in the manufacturer's submission and identified areas requiring clarification, for which the manufacturer provided additional evidence. The relevant patient population was patients aged 6-11 years of age with severe persistent allergic immunoglobulin E-mediated asthma whose condition remained uncontrolled despite best standard care with high-dose inhaled corticosteroids and a long-acting inhaled β₂-agonist. The main clinical effectiveness data were derived from a pre-planned subgroup analysis of a single randomized controlled trial comparing omalizumab plus standard therapy against standard therapy alone. At a 52-week follow-up, the only outcome to show a statistically significant benefit of omalizumab compared with placebo was the number of exacerbations defined as 'clinically significant' [CS] (relative risk [RR] 0.504; 95% CI 0.350, 0.725; p < 0.001). At the ERG's request, the manufacturer provided analyses stratified by baseline exacerbation rate, which indicated the effect of omalizumab on CS exacerbations was statistically significant only for those children with ≥3 exacerbations as baseline. The ERG identified a number of issues relating to the clinical effectiveness results: it was unclear whether the pre-planned subgroup analysis had sufficient power; the definition of CS exacerbation was less severe than that used in UK clinical practice; and the method for imputing exacerbations for those who withdrew from treatment may have underestimated the exacerbation rate. The incremental cost-effectiveness ratio based on the manufacturer's results was considerably above the threshold range stated in the NICE Guide to the Methods of Technology Appraisal. The ERG identified numerous issues relating to the cost-effectiveness results, which included the following: the 10-year time horizon for treatment may exceed that in clinical practice; the assumption of constant exacerbation rates over a lifetime given that adolescence is expected to impact on the severity of asthma; and whether it is appropriate to use health-related quality-of-life data collected in adults for children. The ERG concluded that omalizumab appears to reduce CS exacerbations but there was no evidence of improvement in daily symptoms, CS severe (CSS) exacerbations or hospitalization rates. The main driver of cost effectiveness was the reduction in asthma-related mortality associated with a reduction in CSS exacerbations. As the number of CSS exacerbations avoided was low, as is asthma-related mortality in children, the potential small gain in QALYs associated with omalizumab was not sufficient to compensate for the high treatment cost even under the most favourable scenario analyses. The Appraisal Committee recommended that omalizumab should not be routinely provided for the treatment of severe persistent allergic asthma in children aged 6-11 years.

Diffusion and use of health technology assessment in policy making: what lessons for decentralised healthcare systems?

OBJECTIVE

The Italian National Healthcare System (NHS) is one of the most decentralised systems since the devolution reform approved in 2001. HTA is spreading as an important tool for decision-making processes both at central and local levels. The aims of this study were to review the state of the health technology assessment (HTA) programmes in Italy - with a focus on regional and central initiatives - and to discuss consequences of a multi-level structure of HTA agencies in highly regionalised healthcare systems.

METHODS

Our method combined documentary review with interviews. We reviewed scientific literature about HTA's activities in decentralised systems, legislative and administrative documents from national as well as regional authorities. Semi-structured interviews were conducted with 18 key individuals associated with HTA both at the national and regional levels. Data on HTA programmes implemented or under development in nine regions were collected and analysed according to key principles for the improved conduct of health technology assessments for resource allocation decisions.

RESULTS

HTA is in the early stage of development in Italy, although with great heterogeneity across regions. The National Agency for Health Services has certainly contributed to HTA diffusion through supporting and training activities. However, the multi-level structure of HTA in Italy has not yet provided full coordination and harmonisation of practices and outcomes across the country, with a consequent exacerbate inequality of access to services and technologies.

CONCLUSIONS

There is probably need to rethink the multi-layer organizational framework of HTA in Italy by leveraging on current knowledge and efficient redistribution of activities across regions. We would advise for different jurisdictions playing different roles while achieving similar health outcomes for their patients, rather than jurisdictions aiming at doing exactly the same things resulting in unequal access to healthcare service provision.

The role of health technology assessment in coverage decisions on newborn screening

OBJECTIVES

The role and impact of health technology assessment (HTA) in health policy has been widely discussed. Researchers have started to analyze how decisions on coverage of new technologies are made. Although the involvement of HTA may be an indicator of a well established decision process, this hypothesis requires validation. Also, it is not known whether HTA involvement is associated with other characteristics of decision making like participation or transparency. The primary objective of this study was to develop and test statements on the association between the publication of an HTA and coverage decision making for newborn screening tests in European Union countries.

METHODS

Five statements were defined on the relative role of HTA during the steps of decision processes: trigger, participation, publication, assessment, and appraisal. For this purpose, data on twenty-two decision processes in the area of newborn screening across Europe were analyzed, defined as a coverage decision for a given disorder in a specific country. Decision processes were compared by whether the decision was accompanied by the publication of an HTA report. To test differences, nonparametric statistical tests were used.

RESULTS

The decision steps of trigger, participation and publication differed between the HTA and the non-HTA groups. No clear association between HTA and assessment methods in coverage decision making was identified.

CONCLUSIONS

It appeared that there is an association between HTA and coverage decision processes that are more explicit, inclusive, and transparent. It is unclear whether HTA is associated with formal evidence reviews and economic evaluations.

A new reimbursement system for innovative pharmaceuticals combining value-based and free market pricing

Sweden has experienced a national value-based pricing (VBP) system for innovative outpatient drugs operated by the Pharmaceutical Benefits Board - LFN (now called the Dental and Pharmaceutical Benefits agency - TLV) since 2002. VBP has the character of a monopoly system, leading to reimbursement decisions where usage of new medicines is limited to subgroups and not the population for which the drug is approved. VBP relies on a broad societal perspective, encouraging innovations by signaling to firms that value-adding treatments are demanded. However, the VBP system is operated without a drug budget responsibility. The budget responsibility lies at the regional level, not operating VBP, thus an intrinsic conflict is built into the system. The aim of this article is to suggest a modification to the current reimbursement system in Sweden where payment for pharmaceuticals is split between the regional and national levels. The system is expected to make new innovative pharmaceuticals accessible to a larger number of patients and provide more consumer surplus without reducing the producer surplus. In short, the county councils pay the marginal cost of production while the state pays for the innovation.

CAN WE RELIABLY BENCHMARK HEALTH TECHNOLOGY ASSESSMENT ORGANIZATIONS?

Objectives: In recent years, there has been growth in the use of health technology assessment (HTA) for making decisions about the reimbursement, coverage, or guidance on the use of health technologies. Given this greater emphasis on the use of HTA, it is important to develop standards of good practice and to benchmark the various HTA organizations against these standards.

Methods: This study discusses the conceptual and methodological challenges associated with benchmarking HTA organizations and proposes a series of audit questions based on a previously published set of principles of good practice.

Results and Conclusions: It is concluded that a benchmarking exercise would be feasible and useful, although the question of who should do the benchmarking requires further discussion. Key issues for further research are the alternative methods for weighting the various principles and for generating an overall score, or summary statement of adherence to the principles. Any weighting system, if developed, would need to be explored in different jurisdictions to assess the extent to which the relative importance of the principles is perceived to vary. Finally, the development and precise wording of the audit questions requires further study, with a view to making the questions as unambiguous as possible, and the reproducibility of the assessments as high as possible.

Differences among formulary submission guidelines: implications for health technology assessment

OBJECTIVES

This article provides a detailed understanding of the differences in selected formulary submission guidelines supplied by various health technology assessment (HTA) agencies and indicates how these differences can impact the evidence base used to populate the HTA.

METHODS

Detailed summaries of the recommended methods for evidence generation, organized by topic areas relevant for clinical and economic data, for twelve countries in Europe, North America, and Australia where HTA processes are well developed were prepared. Using these summaries, we provide examples of the likely impact these differences in recommended methods could have on the evidence base used to evaluate new health technologies.

RESULTS

Areas where recommendations differed included methodologies for systematic literature reviews (e.g., preferred databases and study designs for inclusion); selection of appropriate comparators; guidance on critical appraisal and synthesis of clinical evidence; appropriate sources for health value measures, resource use, and cost data; and approaches to uncertainty analyses. Performing literature searches that capture all relevant studies and then creating subsets of the literature based on a listing of country-specific requirements could allow for direct comparison of the evidence bases associated with the different guidelines.

CONCLUSIONS

If the formulary submission guidelines were followed as written, different (although overlapping) bodies of evidence likely would be generated for each country, which could contribute to disparate assessments and recommendations. This comparison of the formulary submission guidelines could contribute to an understanding of why clinical and reimbursement decisions vary across countries.

Bridging trial and decision: a checklist to frame health technology assessments for resource allocation decisions

OBJECTIVE

Health technology assessments (HTAs) intend to inform real-world decisions. They often draw on data from explanatory trials and hence are not always applicable to the decision problem. HTAs may therefore not meet the needs of decision makers. Our objective was to develop and apply a checklist to: 1) systematically frame HTAs in a way that they are applicable to the decision problem; and 2) assess if a decision problem can be informed by an available HTA.

METHODS

We reviewed published literature to identify factors that should be considered when framing HTAs for resource allocation decisions. The checklist was finalized in collaboration with clinicians and policy makers. We applied the checklist to the economic evaluation of trastuzumab in early breast cancer. We defined a reference case and for each study, retrieved through a systematic review, we examined if each factor was explicitly considered.

RESULTS

A checklist was developed with 11 factors (e.g., clinical practice, consequences, and patient use). In the case of trastuzumab, most factors were considered by the 11 retrieved economic evaluations. Two factors, being the inclusion of all relevant comparators and professional use, were considered by none of the studies.

CONCLUSIONS

We developed a comprehensive checklist with 11 factors to frame HTAs and to assess the applicability of HTAs to resource allocation decisions. Economic evaluations on trastuzumab considered some of these factors, but overlooked others. The proposed checklist assists in systematically considering all factors in developing the conceptual model of an HTA, to make HTAs better reflect the decision problem.

Early dialogue between the developers of new technologies and pricing and reimbursement agencies: a pilot study

It is common practice for developers of new health care technologies to engage in early dialogue with the major regulatory agencies; such discussions frequently center around the proposed clinical trial designs to support the registration of new interventions and suggestions on their improvement. Pricing and reimbursement agencies are increasingly using the results from health technology assessments to inform their decision making for new technologies. Such assessments are invariably underpinned by the phase 3 clinical trial evidence which may not provide answers to the key questions. Technology developers are beginning to realize that direct, early dialogue on the evidence requirements of the major pricing and reimbursement agencies, before phase 3 clinical trial designs for their key development compounds have been finalized, may be beneficial. This article reports on the pioneering efforts of one technology developer in seeking early dialogue with seven pricing and reimbursement agencies in five countries globally in 2007-2008 on their likely evidence requirements for a new oral treatment for patients with chronic plaque psoriasis. The pilot project demonstrated that a feasible process of early dialogue could be established, through a face-to-face meeting with prior circulation of a briefing book. Although there was some variation in the advice the similarities far outweighed the differences. More experience of early dialogue needs to be accumulated, involving a wider range of pricing and reimbursement agencies and compounds. The conclusion of this study, however, was that early dialogue can be a worthwhile process for all parties and can lead to a common understanding about evidence development for market access.

When the decision is what to decide: using evidence inventory reports to focus health technology assessments

OBJECTIVES

Health systems frequently make decisions regarding acquisition and use of new technologies. It is desirable to base these decisions on clinical evidence, but often these technologies are used for multiple indications and evidence of effectiveness for one indication does not prove effectiveness for all. Here, we describe two examples of evidence inventory reports that were performed for the purposes of identifying how much and what type of published clinical evidence was available for a given technology, and the contexts in which those technologies were studied.

METHODS

The evidence inventory reports included literature searches for systematic reviews and health technology assessment (HTA) reports, and systematic searches of the primary literature intended to count and categorize published clinical studies. The reports did not include analysis of the primary literature.

RESULTS

The inventory reports were completed in 3 to 4 days each and were approximately ten pages in length, including references. Reports included tables listing the number of reported studies by specific indication for use, and whether or not there were randomized trials. Reports also summarized findings of existing systematic reviews and HTA reports, when available. Committees used the inventory reports to decide for which indications they wanted a full HTA report.

CONCLUSIONS

Evidence inventory reports are a form of rapid HTA that can give decision makers a timely understanding of the available evidence upon which they can base a decision. They can help HTA providers focus subsequent reports on topics that will have the most influence on healthcare decision making.

What aspects of the health technology assessment process recommended by international health technology assessment agencies received the most attention in Poland in 2008?

Objectives: The primary objective of this study was to determine the extent to which international standards on transparency and quality are met by the health technology assessment (HTA) process in Poland. A secondary objective is to describe the outcomes of the HTA process and their associated factors.

Methods: All published online HTA appraisal and meeting proceedings on pharmaceutical products in 2008 were reviewed using a score card developed from international checklists recommended by INAHTA and ECHTA.

Results: The sixty-nine reports reviewed showed that five of nine transparency standards and six of eight quality standards were usually met by the HTA reports. Areas for improvement for transparency include inputs from external stakeholders, availability of English summaries, conclusions, implications of results, and suggested program of action. Areas of improvement for quality include appropriateness of target population and comparator/s, sufficiency of evidence on efficacy and safety, methodological rigor, economic model assumptions, and adaptation to the Polish setting. A consideration of the ethical and social consequences to the healthcare system must also be strengthened.

Conclusions: The study demonstrates that the incorporation and implementation of the HTA appraisal process in Poland has been successful. HTA appraisal reports in Poland have considered most of the international standards of transparency and quality. Recommendations for both HTA users and doers are forwarded for the improvement of the HTA process in the Polish setting.

HTA principles need rigorous review

In the article “Are key principles for improved health technology assessment supported and used by health technology assessment organizations?” by Neumann et al. (3), the authors set out to evaluate the quality of processes used by health technology assessment (HTA) programs based on a set of principles they published in this journal previously (2). We believe that portions of the current article, as they relate to our own organization, are misleading or inaccurate, and welcome this opportunity to correct the inaccuracies.

EBM, HTA, and CER: clearing the confusion

CONTEXT

The terms evidence-based medicine (EBM), health technology assessment (HTA), comparative effectiveness research (CER), and other related terms lack clarity and so could lead to miscommunication, confusion, and poor decision making. The objective of this article is to clarify their definitions and the relationships among key terms and concepts.

METHODS

This article used the relevant methods and policy literature as well as the websites of organizations engaged in evidence-based activities to develop a framework to explain the relationships among the terms EBM, HTA, and CER.

FINDINGS

This article proposes an organizing framework and presents a graphic demonstrating the differences and relationships among these terms and concepts.

CONCLUSIONS

More specific terminology and concepts are necessary for an informed and clear public policy debate. They are even more important to inform decision making at all levels and to engender more accountability by the organizations and individuals responsible for these decisions.

Comparative effectiveness without head-to-head trials: a method for matching-adjusted indirect comparisons applied to psoriasis treatment with adalimumab or etanercept

The absence of head-to-head trials is a common challenge in comparative effectiveness research and health technology assessment. Indirect cross-trial treatment comparisons are possible, but can be biased by cross-trial differences in patient characteristics. Using only published aggregate data, adjustment for such biases may be impossible. Although individual patient data (IPD) would permit adjustment, they are rarely available for all trials. However, many researchers have the opportunity to access IPD for trials of one treatment, a new drug for example, but only aggregate data for trials of comparator treatments. We propose a method that leverages all available data in this setting by adjusting average patient characteristics in trials with IPD to match those reported for trials without IPD. Treatment outcomes, including continuous, categorical and censored time-to-event outcomes, can then be compared across balanced trial populations. The proposed method is illustrated by a comparison of adalimumab and etanercept for the treatment of psoriasis. IPD from trials of adalimumab versus placebo (n = 1025) were re-weighted to match the average baseline characteristics reported for a trial of etanercept versus placebo (n = 330). Re-weighting was based on the estimated propensity of enrolment in the adalimumab versus etanercept trials. Before matching, patients in the adalimumab trials had lower mean age, greater prevalence of psoriatic arthritis, less prior use of systemic treatment or phototherapy, and a smaller mean percentage of body surface area affected than patients in the etanercept trial. After matching, these and all other available baseline characteristics were well balanced across trials. Symptom improvements of ≥75% and ≥90% (as measured by the Psoriasis Area and Severity Index [PASI] score at week 12) were experienced by an additional 17.2% and 14.8% of adalimumab-treated patients compared with the matched etanercept-treated patients (respectively, both p < 0.001). Mean percentage PASI score improvements from baseline were also greater for adalimumab than for etanercept at weeks 4, 8 and 12 (all p < 0.05). Matching adjustment ensured that this indirect comparison was not biased by differences in mean baseline characteristics across trials, supporting the conclusion that adalimumab was associated with significantly greater symptom reduction than etanercept for the treatment of moderate to severe psoriasis.