Confirmatory versus explorative endpoint analysis: Decision-making on the basis of evidence available from market authorization and early benefit assessment for oncology drugs

Strengthening the Interface of Evidence-Based Decision Making Across European Regulators and Health Technology Assessment Bodies

OBJECTIVES

Access to medicines in Europe depends on a benefit-risk decision taken by regulators and a relative effectiveness assessment performed by health technology assessment bodies (HTABs) to inform, as one element, a reimbursement decision. Although various similarities in evidence needs exist, understanding of their needs is currently suboptimal and therefore the evidence generated does not always meet their needs. Subsequently, delays in decision making can be expected, negatively affecting access. To overcome this, this study reviewed the evidentiary needs of European regulators and HTABs at European level and analyzed how their collaboration can further facilitate optimal evidence generation plans, evidence use, and evidence presentation.

METHODS

Through systematic literature review, expert interviews, and pairwise comparison of assessment reports by the European Medicines Agency and European network for health technology assessment, respective clinical evidence requirements and impact of product-specific collaboration between European Medicines Agency and HTABs were established.

RESULTS

Clinical evidence needs are quite similar but differences exist in comparator choice, preferred efficacy endpoints, and target population. Results of the impact of collaboration to date were mixed: preapproval joint advice procedures were successful and highly valued by all stakeholders; information exchange at the time of regulatory decision is coming together, yet the European Public Assessment Report can be further optimized; and collaboration on postlicensing evidence generation requirements shows potential but needs solidifying.

CONCLUSIONS

These findings demonstrate the potential to further improve the evidence utilization across stakeholders to avoid duplication and streamline decision making, to ultimately improve access to medicines for European patients.

Towards compatibility of EUnetHTA JCA methodology and German HTA: a systematic comparison and recommendations from an industry perspective

OBJECTIVE

The transferability of the EU joint clinical assessment (JCA) reports for pharmaceuticals for the German benefit assessment was evaluated by systematically comparing EU JCA and German clinical assessments (CA) based on established assessment elements for HTA and assessing the potential impact of differences on Federal Joint Committee (Gemeinsamer Bundesausschuss, G-BA) ability to derive the therapeutic added value.

METHODS

Identification of all pharmaceuticals undergoing both, EU JCA and German CA between January 2016-June 2020. Qualitative review and data extraction from the assessments, assessment of methodological differences using a hierarchical model. Recommendations for harmonisation were developed and consented with pharmaceutical industry stakeholders.

RESULTS

Differences with potentially major impact: (1) View on differing treatment algorithms and definition of corresponding subpopulations/respective comparators. (2) Clinical relevance of surrogate/intermediate endpoints. Inclusion of different/surrogate morbidity endpoints resulting in different relative effectiveness conclusions. (3) Tolerance of study interventions not used according to marketing authorisation. (4) Different operationalisation and/or weighting of individual safety endpoints leading to differing relative safety conclusions. Differences with potentially minor impact: (1) Disagreement in risk of bias assessment for overall survival and its robustness against study limitations. (2) Use of patient-reported outcome symptom scales as measurements for health-related quality of life instruments.

CONCLUSION

While many synergies between EU JCA and German CA exist, we identified several aspects in HTA methodology that would benefit of harmonisation and ensure the transferability of future EU JCA to the German HTA process without duplicated evaluation requirements. For those, a set of recommendations was developed.

A decade of early benefit assessment of ophthalmic drugs in Germany: success story or not?

OBJECTIVE

To analyze how ophthalmic drugs fared in the early benefit assessment (EBA) after its introduction in Germany up to 2020 and to quantify its impact on their negotiated prices.

METHODS

Relevant documents were screened and essential content on added benefit outcomes and the underlying evidence was extracted next to pricing information. In addition to descriptive statistics, cross-stakeholder analyses and agreement statistics were implemented.

RESULTS

Thirteen completed EBA were identified involving eight drugs. Only four drugs (30.8%) received an added benefit. The OR for no added benefit of ophthalmic drugs versus all other drugs was 2.971 (0.902-9.781). The agreement between manufacturers' claims and decision-maker appraisals is fair (kappa 0.435). In all cases, evidence was derived for RCTs, but for different reasons, not all of them allowed direct comparisons with the comparator as defined by the decision-maker. The negotiated rebates on manufacturer's selling prices varied from 6.8% up to 47.4%. Nevertheless, the rebates for ophthalmic drugs (median 14.5%) were lower than those for all negotiated drugs (median 24%).

CONCLUSION

Over the past decade, the EBA of ophthalmic drugs was not necessarily a success story, but in most of the cases, the drugs were successful in the market.

Bridging the gap: Can International Consortium of Health Outcomes Measurement standard sets align outcomes accepted for regulatory and health technology assessment decision-making of oncology medicines

Standard outcome sets developed by the International Consortium for Health Outcomes Measurement (ICHOM) facilitate value-based health care in healthcare practice and have gained traction from regulators and Health Technology Assessment (HTA) agencies that regularly assess the value of new medicines. We aimed to assess the extent to which the outcomes used by regulators and HTA agencies are patient-relevant, by comparing these to ICHOM standard sets. We conducted a cross-sectional comparative analysis of ICHOM standard sets, and publicly available regulatory and HTA assessment guidelines. We focused on oncology due to many new medicines being developed, which are accompanied by substantial uncertainty regarding the relevance of these treatments for patients. A comparison of regulatory and HTA assessment guidelines, and ICHOM standard sets showed that both ICHOM and regulators stress the importance of disease-specific outcomes. On the other hand, HTA agencies have a stronger focus on generic outcomes in order to allow comparisons across disease areas. Overall, similar outcomes are relevant for market access, reimbursement, and in ICHOM standard sets. However, some differences are apparent, such as the acceptability of intermediate outcomes. These are recommended in ICHOM standard sets, but regulators are more likely to accept intermediate outcomes than HTA agencies. A greater level of alignment in outcomes accepted may enhance the efficiency of regulatory and HTA processes, and increase timely access to new medicines. ICHOM standard sets may help align these outcomes. However, some differences in outcomes used may remain due to the different purposes of regulatory and HTA decision-making.

The impact of additive or substitutive clinical study design on the negotiated reimbursement for oncology pharmaceuticals after early benefit assessment in Germany

BACKGROUND

We analysed the impact of clinical study design for oncological pharmaceuticals on the subsequent price negotiations after early benefit assessment between pharmaceutical companies and the German National Association of Statutory Health Insurance Funds. The analysis was conducted for all oncology pharmaceuticals that underwent the early benefit assessment in Germany since its introduction in 2011 up to September 2016.

METHODS

It was differentiated between additive (new therapy in addition to baseline therapy) and substitutive study designs (baseline therapy to be replaced). The study design was derived from the dossiers of the pharmaceutical companies submitted to the Federal Joint Committee. Subgroup specific costs in case of granted added benefit were calculated as annual therapy costs and compared with the costs of the appropriate comparators to quantify price premiums. Further price influencing factors were analysed in univariate and multivariate regression analysis considering the budget impact for the statutory health insurance as well.

RESULTS

The mean and the median of the additive premiums for substitutive designs (€50,477.68 and €49,841.24) were higher than for additive designs, if the comparator was different to best supportive care (€48,750.00 and €42,820.44). The mean multiplicative premium for the substitutive designs was 15.07 versus 2.29 for the additive designs. EU-Prices and target population size had a significant effect on the reimbursement. The adjusted R-square in the log Premium OLS-regressions reached 0.708 when including all explanatory variables and considering interaction between target population and annual costs of the comparator.

CONCLUSIONS

Study design as an additional important influencing factor of the negotiations next to those stated in the framework agreement was identified and verified. Therefore, study design should be considered by pharmaceutical companies and by decision makers and payers within strategic price planning as a potential predictor. For some specific categories the number of cases was small. Further analyses should be performed when more oncology pharmaceuticals have passed the early benefit assessment.

Different interpretation of additional evidence for HTA by the commissioned HTA body and the commissioning decision maker in Germany: whenever IQWiG and Federal Joint Committee disagree

BACKGROUND

The purpose of this study was to analyse the impact of commissioned addenda by the Federal Joint Committee (FJC) to the HTA body (IQWiG) and their agreement with FJC decisions and to identify potential additional decisive factors of FJC.

METHODS

All available relevant documents up to end of 2017 were screened and essential content extracted. Next to descriptive statistics, differences between IQWiG and FJC were tested and explored by agreement statistics (Cohen's kappa and Fleiss' kappa) and ordinal logistic regression.

RESULTS

Most of the 90 addenda concerned oncological products. In all contingent comparisons, positive changes in added benefit or evidence level on a subpopulation basis (n = 124) prevailed negative ones. Fleiss' ordinal kappa for agreement of assessments, addenda, and appraisals reached a moderate strength for added benefit (0.474, 95%-CI, 0.408-0.540). Overall agreement between addenda and appraisals on a binary nominal basis is poor for added benefit (Cohen's kappa 0.183; 95%-CI: 0.010-0.357) ranging from "less than by chance" (respiratory diseases) to "perfect" (neurological diseases). The OR of the selected regression model showed that i) mortality, ii) unmet need, the positions of iii) the physicians' drug commission and iv) medical societies, and v) the annual therapeutic costs of the appropriate comparative therapy had a high influence on FJC's appraisals deviating from IQWiG's addenda recommendation.

CONCLUSIONS

IQWiG's addenda have a high impact on decision-maker's appraisals offering additional analyses of supplementary evidence submitted by the manufacturers. Nevertheless, the agreement between addenda and appraisals varies, highlighting different decisive factors between IQWiG and FJC.

Methodological problems in the method used by IQWiG within early benefit assessment of new pharmaceuticals in Germany

BACKGROUND

The decision matrix applied by the Institute for Quality and Efficiency in Health Care (IQWiG) for the quantification of added benefit within the early benefit assessment of new pharmaceuticals in Germany with its nine fields is quite complex and could be simplified. Furthermore, the method used by IQWiG is subject to manifold criticism: (1) it is implicitly weighting endpoints differently in its assessments favoring overall survival and, thereby, drug interventions in fatal diseases, (2) it is assuming that two pivotal trials are available when assessing the dossiers submitted by the pharmaceutical manufacturers, leading to far-reaching implications with respect to the quantification of added benefit, and, (3) it is basing the evaluation primarily on dichotomous endpoints and consequently leading to an information loss of usable evidence.

OBJECTIVE

To investigate if criticism is justified and to propose methodological adaptations.

METHODS

Analysis of the available dossiers up to the end of 2016 using statistical tests and multinomial logistic regression and simulations.

RESULTS

It was shown that due to power losses, the method does not ensure that results are statistically valid and outcomes of the early benefit assessment may be compromised, though evidence on favoring overall survival remains unclear. Modifications, however, of the IQWiG method are possible to address the identified problems.

CONCLUSION

By converging with the approach of approval authorities for confirmatory endpoints, the decision matrix could be simplified and the analysis method could be improved, to put the results on a more valid statistical basis.