EMA and NICE Appraisal Processes for Cancer Drugs: Current Status and Uncertainties

Potential impact on cost-effectiveness estimates of using immature survival data: a case study based on transcatheter edge-to-edge repair (TEER) used for patients with severe mitral regurgitation at high surgical risk

OBJECTIVE

To review the survival modelling used in cost-effectiveness studies evaluating an interventional procedure and to discuss implications for decision-makers.

DESIGN

A case study of three economic evaluations that each used immature data from the EVEREST II High Surgical Risk (HSR) Study of transcatheter edge-to-edge repair (TEER) for patients with severe mitral regurgitation (MR) who were at high risk of surgery.

SETTING

Estimation of patient survival in cost-effectiveness studies.

PARTICIPANTS

The EVEREST II HSR Study included 78 patients who had TEER of the mitral valve using the MitraClip device and a retrospectively identified control group of 36 patients who received medical management and were followed up for 12 months. Observed survival (TEER arm only) was updated at 5 years.

RESULTS

Two studies used 12-month observed mortality from EVEREST II HSR to model survival over lifetime horizons. Observed and modelled survival were associated with considerable uncertainty due to short follow-up and small numbers of participants. Modelling control patients' survival required an approximate 10-fold extrapolation based on 12-month observation of only 38 patients. Observed 5-year survival in the TEER group differed from that less mature follow-up suggesting that survival modelling based on shorter follow-up was unsatisfactory. No public domain data for the control group are available beyond 12-month follow-up so meaningful estimates using mature data for both arms are currently not possible. A third study developed survival models using incompletely reported transitions between MR grades in EVEREST II HSR and mortality rates observed for different MR grades derived from a study in an unrelated population.

CONCLUSIONS

Modelling survival in such small samples followed up for only 12 months is associated with great uncertainty, and cost-effectiveness results based on these analyses should be viewed as premature and used cautiously in reimbursement decisions.

A Comparative Analysis of Anticancer Drug Appraisals Including Managed Entry Agreements in South Korea and England

OBJECTIVES

This study aimed to compare appraisal decisions about anticancer drugs between the health technology assessment (HTA) agencies in Korea and England, and investigate whether the decisions and supporting evidence are comparable.

METHODS

This study identified 49 anticancer drugs listed by the Korean Ministry of Health and Welfare between January 2014 and December 2019. Of those, 46 anticancer drugs for 58 indications were included for analysis. Official appraisal documents from both countries for 58 drug-indication pairs were compared and assessed in terms of clinical and economic evidence. Evidence items and their groups for analysis were predefined.

RESULTS

Three-quarters of cases were recommended with managed entry agreements (MEAs) in England and three-fifths in Korea. Finance-based MEA types were most common in both countries. Korean and English authorities made consistent decisions in 48 cases (83%) when classifying decisions as 'recommended' and 'not recommended', while the degree of agreement lowered to 16 cases (28%) when subdividing decisions according to MEA types. When the evidence base was identical, their decisions were more likely to be consistent. Regarding clinical evidence, while the majority of cases referred to the same pivotal studies, differences between the committees' recognized comparators and the appraisal date caused discrepancies in decisions. Economic evidence, including incremental cost-effectiveness ratio (ICER) estimates, was identical in only 12 cases (21%), which contributed to discrepancies.

CONCLUSION

England relies on economic evaluation, with increasing use of data collection agreements, in contrast with Korea's new procedure exempting companies from providing economic evaluation. While there is possibility for international cooperation in the assessment of clinical evidence, transferability issues exist, particularly with regard to economic evidence.

Exploring uncertainty and use of real-world data in the National Institute for Health and Care Excellence single technology appraisals of targeted cancer therapy

OBJECTIVES

Dealing with uncertainty is one of the critical topics in health technology assessment. The greater decision uncertainty in appraisals, the less clear the clinical- and cost-effectiveness of the health technology. Although the development of targeted cancer therapies (TCTs) has improved patient health care, additional complexity has been introduced in drug appraisals due to targeting more specific populations. Real-world data (RWD) are expected to provide helpful information to fill the evidence gaps in appraisals. This study compared appraisals of TCTs with those of non-targeted cancer therapies (non-TCTs) regarding sources of uncertainty and reviewed how RWD have been used to supplement the information in these appraisals.

METHODS

This study reviews single technology appraisals (STAs) of oncology medicines performed by the National Institute for Health and Care Excellence (NICE) over 11 years up to December 2021. Three key sources of uncertainty were identified for comparison (generalisability of clinical trials, availability of direct treatment comparison, maturity of survival data in clinical trials). To measure the intensity of use of RWD in appraisals, three components were identified (overall survival, volume of treatment, and choice of comparators).

RESULTS

TCTs received more recommendations for provision through the Cancer Drugs Fund (27.7, 23.6% for non-TCT), whereas similar proportions were recommended for routine commissioning. With respect to sources of uncertainty, the external validity of clinical trials was greater in TCT appraisals (p = 0.026), whereas mature survival data were available in fewer TCT appraisals (p = 0.027). Both groups showed similar patterns of use of RWD. There was no clear evidence that RWD have been used more intensively in appraisals of TCT.

CONCLUSIONS

Some differences in uncertainty were found between TCT and non-TCT appraisals. The appraisal of TCT is generally challenging, but these challenges are neither new nor distinctive. The same sources of uncertainty were often found in the non-TCT appraisals. The uncertainty when appraising TCT stems from insufficient data rather than the characteristics of the drugs. Although RWD might be expected to play a more active role in appraisals of TCT, the use of RWD has generally been limited.

Prevalence of Immature Survival Data for Anti-Cancer Drugs Presented to the National Institute for Health and Care Excellence and Impact on Decision Making

OBJECTIVES

This research aims to explore how often the National Institute for Health and Care Excellence (NICE) uses immature overall survival data to inform reimbursement decisions on cancer treatments, and the implications of this for resource allocation decisions.

METHODS

NICE cancer technology appraisals published between 2015 and 2017 were reviewed to determine the prevalence of using immature survival data. A case study was used to demonstrate the potential impact of basing decisions on immature data. The economic model submitted by the company was reconstructed and was populated first using survival data available at the time of the appraisal, and then using data from an updated data cut published after the appraisal concluded. The incremental cost-effectiveness ratios (ICERs) obtained using the different data cuts were compared. Probabilistic sensitivity analysis was undertaken and expected value of perfect information estimated.

RESULTS

Forty-one percent of NICE cancer technology appraisals used immature data to inform reimbursement decisions. In the case study, NICE gave a positive recommendation for a limited patient subgroup, with ICERs too high in the complete patient population. ICERs were dramatically lower when the final data cut was used, irrespective of the parametric model used to model survival. Probabilistic sensitivity analysis and expected value of perfect information may not have fully characterized uncertainty, because as they did not account for structural uncertainty.

CONCLUSION

Analyses of cancer treatments using immature survival data may result in incorrect estimates of survival benefit and cost-effectiveness, potentially leading to inappropriate funding decisions. This research highlights the importance of revisiting past decisions when updated data cuts become available.

Joining the Dots: Linking Disconnected Networks of Evidence Using Dose-Response Model-Based Network Meta-Analysis

BACKGROUND

Network meta-analysis (NMA) synthesizes direct and indirect evidence on multiple treatments to estimate their relative effectiveness. However, comparisons between disconnected treatments are not possible without making strong assumptions. When studies including multiple doses of the same drug are available, model-based NMA (MBNMA) presents a novel solution to this problem by modeling a parametric dose-response relationship within an NMA framework. In this article, we illustrate several scenarios in which dose-response MBNMA can connect and strengthen evidence networks.

METHODS

We created illustrative data sets by removing studies or treatments from an NMA of triptans for migraine relief. We fitted MBNMA models with different dose-response relationships. For connected networks, we compared MBNMA estimates with NMA estimates. For disconnected networks, we compared MBNMA estimates with NMA estimates from an "augmented" network connected by adding studies or treatments back into the data set.

RESULTS

In connected networks, relative effect estimates from MBNMA were more precise than those from NMA models (ratio of posterior SDs NMA v. MBNMA: median = 1.13; range = 1.04-1.68). In disconnected networks, MBNMA provided estimates for all treatments where NMA could not and were consistent with NMA estimates from augmented networks for 15 of 18 data sets. In the remaining 3 of 18 data sets, a more complex dose-response relationship was required than could be fitted with the available evidence.

CONCLUSIONS

Where information on multiple doses is available, MBNMA can connect disconnected networks and increase precision while making less strong assumptions than alternative approaches. MBNMA relies on correct specification of the dose-response relationship, which requires sufficient data at different doses to allow reliable estimation. We recommend that systematic reviews for NMA search for and include evidence (including phase II trials) on multiple doses of agents where available.

Economic Evaluations of Anticancer Drugs Based on Medico-Administrative Databases: A Systematic Literature Review

BACKGROUND

Oncology is among the most active therapeutic fields in terms of new drug development projects, with increasingly expensive drugs. The expected clinical benefit and cost effectiveness of these treatments in clinical practice have yet to be fully confirmed. Health medico-administrative databases may be useful for assessing the value of anticancer drugs with real-world data.

OBJECTIVE

The objectives of our systematic literature review (SLR) were to analyse economic evaluations of anticancer drugs based on health medico-administrative databases, to assess the quality of these evaluations, and to identify the inputs from such databases that can be used in economic evaluations of anticancer drugs.

METHODS

We performed an SLR by using PubMed and Web of Science articles published from January 2008 to January 2019. The search strategy focused on anticancer drug cost-effectiveness analyses (CEAs)/cost-utility analyses (CUAs) that were entirely based on medico-administrative databases. The review reported the main choices of economic evaluation methods in the analyses. The quality of the articles was assessed using the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) and risk of bias assessment checklists.

RESULTS

Of the 306 records identified in PubMed, 12 articles were selected, and one additional article was identified through Web of Science. Ten of the 13 articles were CEAs and three were CUAs. Most of the analyses were carried out in North America (n = 11). The economic metric used was the cost per life-year gained (n = 10) or cost per quality-adjusted life-year (n = 3). Reporting of the target analysis population and strategies in the articles was in agreement with the CHEERS guidelines. The structural assumptions underpinning the economic models displayed the poorest reporting quality among the items analysed. Representativeness bias (n = 11) and the issue of censored medical costs (n = 8) were the most frequently analysed risks.

CONCLUSION

A comparison of the economic results was not relevant due to the high heterogeneity of the selected studies. Our SLR highlighted the benefits and pitfalls related to the use of medico-administrative databases in the economic evaluations of anticancer drugs.

Development and Validation of the TRansparent Uncertainty ASsessmenT (TRUST) Tool for Assessing Uncertainties in Health Economic Decision Models

BACKGROUND

An increasing number of technologies are obtaining marketing authorisation based on sparse evidence, which causes growing uncertainty and risk within health technology reimbursement decision making. To ensure that uncertainty is considered and addressed within health technology assessment (HTA) recommendations, uncertainties need to be identified, included in health economic models, and reported.

OBJECTIVE

Our objective was to develop the TRansparent Uncertainty ASsessmenT (TRUST) tool for systematically identifying, assessing, and reporting uncertainties in decision models, with the aim of making uncertainties and their impact on cost effectiveness more explicit and transparent.

METHODS

TRUST was developed by drawing on the uncertainty and risk assessment literature. To develop and validate this tool, we conducted HTA stakeholder discussion meetings and interviews and applied it in six real-world HTA case studies in the Netherlands and the UK.

RESULTS

The TRUST tool enables the identification and categorisation of uncertainty according to its source (transparency issues, methodology issues, and issues with evidence: imprecision, bias and indirectness, and unavailability) in each model aspect. The source of uncertainty determines the appropriate analysis. The impact of uncertainties on cost effectiveness is also assessed. Stakeholders found using the tool to be feasible and of value for transparent uncertainty assessment. TRUST can be used during model development and/or model review.

CONCLUSION

The TRUST tool enables systematic identification, assessment, and reporting of uncertainties in health economic models and may contribute to more informed and transparent decision making in the face of uncertainty.

Accessibility of cancer drugs in Switzerland: Time from approval to pricing decision between 2009 and 2018

BACKGROUND

Approved drugs must be included on the so-called "special list" (SL) by the Federal Office for Public Health (FOPH) to be reimbursed by the social health insurance in Switzerland. The FOPH decides whether a drug may be included on SL and if so, negotiates the maximum price with the manufacturer. Time period between approval and inclusion on SL is important to evaluate accessibility of patients to drugs.

METHODS

We identified all approved cancer drugs for first indication between 2009-2018 in Switzerland and determined whether they have been included on SL, and if so, how many days passed between approval and reimbursement, i.e., inclusion on SL. Descriptive statistical analysis was performed by using R.

RESULTS

70 cancer drugs have been approved between 2009 and 2018. Among this sample, 56 (80 %) are on SL. Average time from drug approval to inclusion date on SL increased from 234 days in 2009 to 463 days in 2018, with an average time of 352 days over the full analysis period.

CONCLUSION

Time period between approval and inclusion on SL has prolonged over the past years. This impedes patients' access to cancer drugs. Prioritizing HTA and price negotiation of cancer drugs with high clinical benefit or implement a regulation that sets a maximum time period for HTA and price negotiation are possible policy implications.

Pembrolizumab for Treating Relapsed or Refractory Classical Hodgkin Lymphoma: An Evidence Review Group Perspective of a NICE Single Technology Appraisal

As part of its Single Technology Appraisal (STA) process, the National Institute for Health and Care Excellence (NICE) invited the manufacturer (Merck Sharp & Dohme; MSD) of pembrolizumab (Keytruda®) to submit evidence of its clinical and cost effectiveness for the treatment of patients with relapsed or refractory classical Hodgkin lymphoma (RRcHL) who did not respond to treatment with brentuximab vedotin. Kleijnen Systematic Reviews Ltd, in collaboration with Maastricht University Medical Centre+, was commissioned to act as the independent Evidence Review Group (ERG). The ERG produced a detailed review of the evidence for the clinical and cost effectiveness of the technology, based on the company's submission to NICE. According to the NICE scope, pembrolizumab was compared with single or combination chemotherapy. Comparisons were undertaken in two populations: patients who did and did not receive prior autologous stem cell transplant (autoSCT; populations 1 and 2, respectively). Despite it having been recommended by NICE in population 1 at the time the ERG received the company submission, nivolumab was not included as a comparator. No studies directly comparing pembrolizumab and its comparators were identified. One ongoing, single-arm study of the efficacy and safety of pembrolizumab (KEYNOTE-087) and one comparative observational study (Cheah et al., 2016) were used to inform the comparative effectiveness of pembrolizumab and standard of care (SoC), using indirect comparisons in both populations. Almost all analyses showed significant PFS and overall response rate benefits for pembrolizumab versus SoC, but due to being based on indirect comparison, were likely to contain systematic error. The economic evaluation therefore suffered from substantial uncertainty in any estimates of cost effectiveness. Furthermore, there was a lack of evidence on the uptake and timing of allogeneic stem cell transplant, and alternative assumptions had a significant impact on cost effectiveness. Immature survival data from KEYNOTE-087 exacerbated this issue and necessitated the use of alternative data sources for longer-term extrapolation of survival. Some issues identified in the company's analyses were amended by the ERG. The revised ERG deterministic base-case incremental cost-effectiveness ratios based on the company's second Appraisal Consultation Document response for pembrolizumab versus SoC (with a commercial access agreement) for populations 1 and 2 were £54,325 and £62,527 per quality-adjusted life-year gained, respectively. There was substantial uncertainty around these ICERs, especially in population 2. NICE did not recommend pembrolizumab as an option for treating RRcHL in population 1, but recommended pembrolizumab for use within the Cancer Drugs Fund in population 2.

Obinutuzumab in Combination with Chemotherapy for the First-Line Treatment of Patients with Advanced Follicular Lymphoma : An Evidence Review Group Evaluation of the NICE Single Technology Appraisal

The National Institute for Health and Care Excellence (NICE), as part of the institute's single technology appraisal (STA) process, invited the company that makes obinutuzumab (Roche Products Limited) to submit evidence of the clinical and cost effectiveness of the drug in combination with chemotherapy, with or without obinutuzumab as maintenance therapy for adult patients with untreated, advanced follicular lymphoma (FL) in the UK. Kleijnen Systematic Reviews Ltd (KSR), in collaboration with Erasmus University Rotterdam, was commissioned to act as the Evidence Review Group (ERG). This paper describes the company's submission, the ERG review, and NICE's subsequent decisions. The clinical evidence was derived from two phase III, company-sponsored, randomised, open-label studies. Most evidence on obinutuzumab was based on the GALLIUM trial that compared obinutuzumab in combination with chemotherapy as induction followed by obinutuzumab maintenance monotherapy with rituximab in combination with chemotherapy as induction followed by rituximab maintenance monotherapy in previously untreated patients with FL (grades 1-3a). Long-term clinical evidence was based on the PRIMA trial, studying the benefit of two years of rituximab maintenance after first-line treatment in patients with FL. The cost-effectiveness evidence submitted by the company relied on a partitioned survival cost-utility model, implemented in Microsoft® Excel. The base-case incremental cost-effectiveness ratio (ICER) presented in the company submission was <£20,000 per quality-adjusted life-year (QALY) gained. Although the ERG concluded that the economic model met the NICE reference case to a reasonable extent, some errors were identified and several assumptions made by the company were challenged. A new base-case scenario produced by the ERG suggested an ICER that was higher than the company base case, but still below £30,000 per QALY gained. However, some ERG scenario analyses were close to or even above the threshold. This was the case in particular for assuming a treatment effect that did not extend beyond trial follow-up. These results led to an initial negative recommendation by the appraisal committee. Subsequently, the company submitted a revised base case focusing on patients at intermediate or high risk of premature mortality. Simultaneously, a further price discount for obinutuzumab was granted. In addition to the company's revised base case, the ERG suggested a restriction of the treatment effect to 5 years and implemented biosimilar uptake and cheaper prices for rituximab. All of these adjustments did not exceed £30,000 per QALY gained and therefore the use of obinutuzumab for patients with advanced FL and a Follicular Lymphoma International Predictive Index (FLIPI) score of two or more could be recommended.

The Cancer Drugs Fund in Practice and Under the New Framework

BACKGROUND

The Cancer Drugs Fund (CDF) was established in 2010 to improve access to treatments not routinely available. Having widely overspent, stricter budgeting rules were introduced in 2016. The CDF can now only include treatments with potential to be cost effective once sufficient data are collected.

OBJECTIVES

Our objective was to explore the process and criteria used for consideration of treatments under the new CDF framework and to describe the extent of evidence collection.

METHODS

We identified CDF list, UK National Institute for Health and Care Excellence (NICE) and Scottish Medicines Consortium documents (10 May 2018). Data were collected on drugs and indications, reasons for inclusion in the CDF, data collection, incremental cost-effectiveness ratios (ICERs), and corresponding recommendations for Scotland.

RESULTS

In total, 12 drugs were listed on the CDF in 17 indications, 12 of which were considered end-of-life care. The most common cancers were non-small-cell lung (n = 4), urothelial (n = 3), lymphocytic leukaemia (n = 2) and multiple myeloma (n = 2). The companies' ICERs were generally lower than those from the committee and the evidence review group. Drugs were included in the CDF for 6-42 months, with the majority included for ≥18 months. Data were frequently collected on overall survival (n = 16) and progression-free survival (n = 5) using NHS systems and, frequently, ongoing trials.

CONCLUSIONS

Data collection frequently included overall survival and exceeded the 2 years recommended in the CDF strategy. It appears the CDF is allowing patients access to drugs long before they may become available for routine use. Our results are limited by the availability of published information and the small dataset.

Nivolumab for Treating Metastatic or Unresectable Urothelial Cancer: An Evidence Review Group Perspective of a NICE Single Technology Appraisal

As part of its single technology appraisal (STA) process, the National Institute for Health and Care Excellence (NICE) invited the manufacturer (Bristol-Myers Squibb) of nivolumab (Opdivo^®) to submit evidence of its clinical and cost effectiveness for metastatic or unresectable urothelial cancer. Kleijnen Systematic Reviews Ltd, in collaboration with Maastricht University Medical Centre+, was commissioned to act as the independent Evidence Review Group (ERG), which produced a detailed review of the evidence for the clinical and cost effectiveness of the technology, based on the company's submission to NICE. Nivolumab was compared with docetaxel, paclitaxel, best supportive care and retreatment with platinum-based chemotherapy (cisplatin plus gemcitabine, but only for patients whose disease has had an adequate response in first-line treatment). Two ongoing, phase I/II, single-arm studies for nivolumab were identified, but no studies directly compared nivolumab with any specified comparator. Evidence from directly examining the single arms of the trial data indicated little difference between the outcomes measured from the nivolumab and comparator studies. A simulated treatment comparison (STC) analysis was used in an attempt to reduce the bias induced by naïve comparison, but there was no clear evidence that risk of bias was reduced. Multiple limitations in the STC were identified and remained. The effect of an analysis based on different combinations of covariates in the prediction model remains unknown. The ERG's concerns regarding the economic analysis included the use of a non-established response-based survival analysis method, which introduced additional uncertainty. The use of time-dependent hazard ratios produced overfitting and was not represented in the probabilistic sensitivity analysis. The use of a treatment stopping rule to cap treatment cost left treatment effectiveness unaltered. A relevant comparator was excluded from the base-case analysis. The revised ERG deterministic base-case incremental cost-effectiveness ratios based on the company's Appraisal Consultation Document response were £58,791, £78,869 and £62,352 per quality-adjusted life-year gained versus paclitaxel, docetaxel and best supportive care, respectively. Nivolumab was dominated by cisplatin plus gemcitabine in the ERG base case. Substantial uncertainties about the relative treatment effectiveness comparing nivolumab against all comparators remained. NICE did not recommend nivolumab, within its marketing authorisation, as an option for treating locally advanced, unresectable or metastatic urothelial carcinoma in adults who have had platinum-containing therapy, and considered that nivolumab was not suitable for use within the Cancer Drugs Fund.