Do different clinical evidence bases lead to discordant health-technology assessment decisions? An in-depth case series across three jurisdictions

Appraisals by Health Technology Assessment Agencies of Economic Evaluations Submitted as Part of Reimbursement Dossiers for Oncology Treatments: Evidence from Canada, the UK, and Australia

Publicly funded healthcare systems, including those in Canada, the United Kingdom (UK), and Australia, often use health technology assessment (HTA) to inform drug reimbursement decision-making, based on dossiers submitted by manufacturers, and HTA agencies issue publicly available reports to support funding recommendations. However, the level of information reported by HTA agencies in these reports may vary. To provide insights on this issue, we describe and assess the reporting of economic methods in recent oncology HTA recommendations from the Canadian Agency for Drugs and Technologies in Health (CADTH), National Institute for Health and Care Excellence (NICE), and Pharmaceutical Benefits Advisory Committee (PBAC). Publicly available HTA recommendations and reports for oncology drugs issued by CADTH over a 2-year period, 2019-2020, were identified and compared with the corresponding HTA documents from NICE and the PBAC. Reporting of key model characteristics and attributes, survival analysis methods, methodological criticisms, and re-assessment of the economic results were characterized using descriptive statistics. Dichotomous differences in the methodological criticisms observed between the three agencies were assessed using Cochran's Q tests and substantiated using pairwise McNemar tests. Chi-squared tests were used to assess the dichotomous differences in the reporting of methods and explore the potential relationships between categorical variables, where appropriate. HTAs published by CADTH, NICE, and the PBAC consistently reported a broad spectrum of descriptive information on the economic models submitted by manufacturers. While common economic evaluation attributes were well-reported across the three HTA agencies, significant differences in the reporting of survival analysis methods and methodological criticisms were observed. NICE consistently reported more comprehensive information, compared to either CADTH or PBAC. Despite these differences, broadly similar recommendation rates were observed between CADTH and NICE. The PBAC was found to be more restrictive. Based on our 2-year sample of oncology, the HTAs published by CADTH matched with the corresponding HTAs from NICE and PBAC; we observed important variations in the reporting of economic evidence, especially technical aspects, such as survival analysis, across the three agencies. In addition to guidelines for HTA submissions by manufacturers, the community of HTA agencies should also have common standards for reporting the results of their assessments, though the information and opinions reported may differ.

Non-Price-Related Determinants of Value and Access for Novel Non-small Cell Lung Cancer Treatments: A Cross-Country Review of HTA Decision Making

INTRODUCTION

Access and funding for newly approved treatments for non-small cell lung cancer (NSCLC) are often dependent on Health Technology Assessment (HTA) involving cost-effectiveness analysis. Whilst methods used by HTA agencies share many similarities, final decisions may differ. This may be the result, not just of price considerations, but also of variation in value judgements by different agencies. The aim of this study was to review international HTA evaluations to identify determinants of value and access for NSCLC treatments.

METHODS

A targeted review and analysis was undertaken of published HTAs for NSCLC across HTA agencies in six countries (Australia, Canada, England, France, Ireland and Scotland). Analysis of extracted data consisted of three stages: descriptive analysis, bivariate analysis and multivariable analysis.

RESULTS

The analysis included 163 HTAs that assessed oncological treatments for NSCLC from 2003 to 2019. The majority of HTA decisions (67.5%) were positive. However, some evidence of heterogeneity in HTA decisions and the factors informing them were identified. The most influential factors included in the multivariate model related to the HTA agency conducting the appraisal, the year of market authorisation, treatment type and the line of treatment.

CONCLUSION

Heterogenous decision-making frameworks can present a challenge to developing HTA submissions. This research contributes to understanding decision-making factors and why countries make different decisions.

From market access to patient access: overview of evidence-based approaches for the reimbursement and pricing of pharmaceuticals in 36 European countries

Background

Coverage decisions determining the benefit baskets of health systems have been increasingly relying on evidence regarding patient benefit and costs. Relevant structures, methodologies, and processes have especially been established for pharmaceuticals but approaches differ. The objective of this work was thus to identify institutions in a broad range of European countries (n = 36) in charge of determining the value of pharmaceuticals for pricing and reimbursement purposes and to map their decision-making process; to examine the different approaches and consider national and supranational possibilities for best practice.

Methods

Institutions were identified through websites of international networks, ministries, and published literature. Details on institutional practices were supplemented with information from institution websites and linked online sources.

Results

The type and extent of information available varied considerably across countries. Different types of public regulatory bodies are involved in pharmaceutical coverage decisions, assuming a range of responsibilities. As a rule, the assessment of scientific evidence is kept structurally separate from its appraisal. Recommendations on value are uniformly issued by specific committees within or commissioned by responsible institutions; these institutions often also act as decision-makers on reimbursement status and level or market price. While effectiveness and costs are important criteria in all countries, the latter are often considered on a case-by-case basis. In all countries, manufacturer applications, including relevant evidence, are used as one of the main sources of information for the assessment.

Conclusion

Transparency of evidence-based coverage decisions should be enhanced. International collaboration can facilitate knowledge exchange, improve efficiency of information production, and strengthen new or developing systems.

Electronic supplementary material

The online version of this article (doi:10.1186/s12961-015-0028-5) contains supplementary material, which is available to authorized users.

A Time-Trend Economic Analysis of Cancer Drug Trials

BACKGROUND

Scientific advances have led to the discovery of novel treatments with high prices. The cost to publicly fund high-cost drugs may threaten the sustainability of drug budgets in different health care systems. In oncology, there are concerns that health-benefit gains are diminishing over time and that the economic evidence to support funding decisions is too limited.

METHODS

To assess the additional costs and benefits gained from oncology drugs over time, we used treatment protocols and efficacy results from U.S. Food and Drug Administration records to calculate cost-effectiveness ratios for drugs approved to treat first- and second-line metastatic or advanced breast, colorectal, and non-small cell lung cancer during the years 1994-2013. We assessed reimbursement recommendations reached by health technology assessment agencies in the U.K., Australia, and Canada.

RESULTS

Cost-effectiveness ratios were calculated for 50 drugs approved by the U.S. regulator. The more recent approvals were often based on surrogate efficacy outcomes and had extremely high costs, often triple the costs of drugs approved in previous years. Over time, the effectiveness gains have increased for some cancer indications; however, for other indications (non-small cell lung and second-line colorectal cancer), the magnitude of gains in effectiveness decreased. Reimbursement recommendations for drugs with the highest cost-effectiveness ratios were the most inconsistent.

CONCLUSION

Evaluation of the clinical benefits that oncology drugs offer as a function of their cost has become highly complex, and for some clinical indications, health benefits are diminishing over time. There is an urgent need for better economic evidence from oncology drug trials and systematic processes to inform funding decisions.

IMPLICATIONS FOR PRACTICE

High-cost oncology drugs may threaten the ability of health care systems to provide access to promising new drugs for patients. In order to make better drug-funding decisions and enable equitable access to breakthrough treatments, discussions in the oncology community should include economic evidence. This study summarizes the extra benefits and costs of newly approved drugs from pivotal trials during the postgenomic era of drug discovery. The reader will gain an appreciation of the need for economic evidence to make better drug-reimbursement decisions and the dynamics at play in today's oncology drug market.

Incremental cost-effectiveness of dobutamine stress cardiac magnetic resonance imaging in patients at intermediate risk for coronary artery disease

Aims

The effectiveness of stress cardiac magnetic resonance (CMR) as a gatekeeper for coronary angiography (CA) has been established. Level five HTA studies according to the hierarchical model of diagnostic test evaluation are not available.

Methods

This cohort study included 1,158 consecutive patients (mean age 63 ± 11 years, 42 % women) presenting at our institution between January 1, 2003 and December 31, 2004 with suspected coronary artery disease (CAD) for an elective CA. The patients were assessed for eligibility and propensity score matching was applied to address selection bias regarding the patients’ allocation to CMR or direct CA. Median patient follow-up was 7.9 years (95 % CI 7.8–8.0 years). The primary effect was calculated as relative survival difference. The cost unit calculation (per patient) at our institute was the source of costs.

Results

Survival was similar in CMR and CA (p = 0.139). Catheterizations ruling out CAD were significantly reduced by the CMR gate-keeper strategy. Patients with prior CMR had significantly lower costs at the initial hospital stay and at follow-up (CMR vs. CA, initial: 2,904€ vs. 3,421€, p = 0.018; follow-up: 2,045€ vs. 3,318€, p = 0.037). CMR was cost-effective in terms of a contribution of 12,466€ per life year to cover a part of the CMR costs.

Conclusion

Stress CMR prior to CA was saving 12,466€ of hospital costs per life year. Lower costs at follow-up suggest sustained cost-effectiveness of the CMR-guided strategy.

Electronic supplementary material

The online version of this article (doi:10.1007/s00392-014-0793-0) contains supplementary material, which is available to authorized users.

Oncology drug health technology assessment recommendations: Canadian versus UK experiences

BACKGROUND

CANADA HAS TWO HEALTH TECHNOLOGY ASSESSMENT (HTA) AGENCIES RESPONSIBLE FOR ONCOLOGY DRUG FUNDING RECOMMENDATIONS: the Institut National d'Excellence en Santé et Services Sociaux (INESSS) for the province of Québec and the pan-Canadian Oncology Drug Review for the rest of Canada. The objective of the research was to review and compare the recommendations of these two agencies alongside an international comparator - the National Institute for Health and Care Excellence (NICE) in the United Kingdom - with respect to their recommendations records and the influence of clinical and cost-effectiveness evidence on the recommendations.

METHODS

Recommendations were identified from the three agencies from January 1, 2002 to June 1, 2013. Recommendations were limited to five cancer sites (lung, breast, colon, kidney, blood) and to metastatic/advanced settings. Descriptive analyses examined the frequency of positive recommendations and factors related to a positive recommendation. For each recommendation, only publicly available information posted on the agency website was used to abstract data.

RESULTS

There was a wide variation in the rate of positive recommendations, ranging from 48% for NICE to 95% for Canada's national process (among the 74% of its recommendations that were publicly posted). Interagency agreement was low, with full agreement for only six of the 14 drugs commonly reviewed by all three agencies. Evidence of a survival gain was not necessary for a positive recommendation; progression-free survival was acceptable. Different approaches were taken when addressing unacceptable cost-effectiveness. NICE was most likely to yield a negative recommendation on these grounds, whereas Canada's national process was most likely to yield a positive recommendation with a required pricing arrangement.

CONCLUSION

In this analysis, the primary reason for the observed divergence between agency recommendations appeared to be the availability of mechanisms in each jurisdiction to address cost-effectiveness subsequent to the HTA assessment process. Furthermore, caution is needed when interpreting cross-agency comparisons between HTA agencies, as recommendations may not correspond directly to subsequent funding decisions and actual patient access. This may be a concern, given the high international profile of assessments conducted by the reviewed HTA agencies.

Surrogate outcomes: experiences at the Common Drug Review

BACKGROUND

Surrogate outcomes are a significant challenge in drug evaluation for health technology assessment (HTA) agencies. The research objectives were to: identify factors associated with surrogate use and acceptability in Canada's Common Drug Review (CDR) recommendations, and compare the CDR with other HTA or regulatory agencies regarding surrogate concerns.

METHODS

Final recommendations were identified from CDR inception (September 2003) to December 31, 2010. Recommendations were classified by type of outcome (surrogate, final, other) and acceptability of surrogates (determined by the presence/absence of statements of concern regarding surrogates). Descriptive and statistical analyses examined factors related to surrogate use and acceptability. For thirteen surrogate-based submissions, recommendations from international HTA and regulatory agencies were reviewed for statements about surrogate acceptability.

RESULTS

Of 156 final recommendations, 68 (44%) involved surrogates. The overall 'do not list' (DNL) rate was 48%; the DNL rate for surrogates was 41% (p = 0.175). The DNL rate was 64% for non-accepted surrogates (n = 28) versus 25% for accepted surrogates (odds ratio 5.4, p = 0.002). Clinical uncertainty, use of economic evidence over price alone, and a premium price were significantly associated with non-accepted surrogates. Surrogates were used most commonly for HIV, diabetes, rare diseases, cardiovascular disease and cancer. For the subset of drugs studied, other HTA agencies did not express concerns for most recommendations, while regulatory agencies frequently stated surrogate acceptance.

CONCLUSIONS

The majority of surrogates were accepted at the CDR. Non-accepted surrogates were significantly associated with clinical uncertainty and a DNL recommendation. There was inconsistency of surrogate acceptability across several international agencies. Stakeholders should consider collaboratively establishing guidelines on the use, validation, and acceptability of surrogates.