Development and validation of COMPASS: clinical evidence of orphan medicinal products - an assessment tool

Oncologic orphan drugs approved in the EU - do clinical trial data correspond with real-world effectiveness?

BACKGROUND

Evaluation of evidence for efficacy of orphan medicinal products (OMPs) for rare malignancies may be hampered by the use of tumor measurements instead of clinical endpoints. This may cause efficacy data to not always match effectiveness in the real-world. We investigated whether an efficacy-effectiveness gap exists for oncologic OMPs and aimed to identify which factors contribute to it. Also, the magnitude of the clinical efficacy of oncologic OMPs was evaluated.

METHODS

We included all oncologic OMPs authorized in the European Union from 2000 to 2017. Pivotal studies were evaluated by means of the European Society for Medical Oncology - Magnitude of Clinical Benefit Scale (ESMO-MCBS). To estimate real-world effectiveness, a literature search was performed to identify post-marketing studies, of which data on overall survival (OS) were extracted. OS of the new OMP was compared with OS data of standard of care. An OS gain of ≥3 months compared to pre-marketing data was considered clinically relevant.

RESULTS

Twenty OMPs were included, of which 5 were authorized based on OS as a primary endpoint. 10 OMPs had post-marketing data available, of which 40% did not show a clinically relevant OS gain in the real world. All OMPs that were studied with OS as primary endpoint in the pivotal study had a clinically relevant OS gain in the real world. Furthermore, all OMPs that had a high ESMO-MCBS score and post-marketing data available, resulted in a clinically relevant OS gain in the real world.

CONCLUSIONS

Although the sample size is small, our results indicate an efficacy-effectiveness gap for oncologic OMPs exists. Significant changes in PFS do not always lead to an increased OS. The use of PFS may be justified, but validation of surrogate endpoints is needed.

Hemophilia Burden of Disease: A Systematic Review of the Cost-Utility Literature for Hemophilia

BACKGROUND

Prophylaxis with clotting factor replacement products is recommended by the Medical and Scientific Advisory Council of the National Hemophilia Foundation as the optimal therapy for the prevention of bleeding episodes in individuals with severe hemophilia A or B (< 1 IU per dL endogenous factor VIII or factor IX activity, respectively). Prophylaxis is associated with an improved health-related quality of life and has been shown to be cost-effective compared with on-demand therapy. However, the overall cost of treatment remains high, particularly among patients with a greater propensity to bleed. The overall value of hemophilia treatments and their associated benefits, measured in quality-adjusted life-years (QALYs), and dollar costs compared with other interventions can be evaluated through the use of cost-utility analyses (CUAs). Previous CUA studies in hemophilia have focused primarily on patients with more severe forms of hemophilia and on prophylaxis compared with on-demand treatment. However, to our knowledge, no studies to date have utilized QALYs as a standardized outcome measure to systematically evaluate the relative cost-effectiveness of current hemophilia treatment options.

OBJECTIVE

To systematically review the CUA literature of hemophilia treatments and demonstrate the challenges in producing cost-utility evidence compared with other rare diseases.

METHODS

We conducted a systematic literature review using the Tufts Medical Center Cost-Effectiveness Analysis Registry and the National Health Service Economic Evaluation Database for English-language CUAs published from 2000 through 2015 with the search terms hemophilia, haemophilia, factor VIII, or factor IX. Two trained reviewers independently reviewed every study to extract relevant data. Incremental cost-effectiveness ratios were converted to 2014 U.S. dollars using exchange rates for currency conversion and the Consumer Price Index to adjust for inflation.

RESULTS

Our search yielded 52 studies, 11 of which met our inclusion criteria. The cost-effectiveness of hemophilia treatments varied widely based on variations in the study designs, including differences in time horizon, discount rates, and medical interventions.

CONCLUSIONS

We found the cost-effectiveness of hemophilia treatments to be broadly comparable to that of other orphan drugs. Improved standardization of future CUA studies will be important for further evaluation of the cost-effectiveness of hemophilia treatments.

DISCLOSURES

This research was funded by Biogen, which provided an unrestricted research grant to the Center for the Evaluation of Value and Risk in Health at Tufts Medical Center. Biogen and Sobi reviewed and provided feedback on the manuscript. The authors had full editorial control of the manuscript and provided final approval of all content. The authors report no conflict of interest regarding the material discussed in this article. Neumann and Chambers are employed at the Center for the Evaluation of Value and Risk in Health at Tufts Medical Center. Thorat was an employee of Center for Evaluation Value and Risk in Health, Tufts Medical Center when the analyses were carried out. Chambers has participated on advisory boards for Sanofi and Astellas Pharma.

Factors Contributing to the Efficacy-Effectiveness Gap in the Case of Orphan Drugs for Metabolic Diseases

Introduction

Authorization of orphan medicinal products (OMPs) is often based on studies with several methodological shortcomings. Hence, data are difficult to interpret and efficacy does not always correspond to real-world effectiveness. We investigated to what extent an efficacy-effectiveness gap exists for OMPs for metabolic diseases and set out to explore which factors contribute to it.

Methods

We included all OMPs for rare metabolic diseases authorized in the EU up to 1 January 2016. Efficacy data were obtained from European Public Assessment Reports, relative effectiveness data from the Dutch National Healthcare Institute website, and real-world effectiveness data from literature and interviews with experts and patients. Efficacy and effectiveness were scored as ‘no effect’, ‘unclear’ or ‘good’ based upon a prespecified scoring system.

Results

We identified 31 authorized OMPs, of which 21 had post-marketing studies available, thus making it possible to score real-world effectiveness. Eight of 21 (38%) OMPs had a ‘good’ real-world effectiveness. The use of a clinical or validated surrogate primary endpoint and a representative study population seemed to be related to good effectiveness in the real world, as were type of marketing authorization, study population and disease prevalence.

Conclusions

This study revealed that less than half of the authorized OMPs are effective in the real world. Since the type of primary endpoint used in the pivotal study seems to be associated with good real-world effectiveness, it is important to agree upon study endpoints through early dialogues among relevant stakeholders.

Electronic supplementary material

The online version of this article (doi:10.1007/s40265-017-0788-z) contains supplementary material, which is available to authorized users.

Clinical evidence for orphan medicinal products-a cause for concern?

Background

The difficulties associated with organising clinical studies for orphan medicinal products (OMPs) are plentiful. Recent debate on the long-term effectiveness of some OMPs, led us to question whether the initial standards for clinical evidence for OMPs, set by the European Medicines Agency (EMA) at the time of marketing authorization, are too low. Therefore, the aim of this study was to quantitatively evaluate the characteristics and quality of clinical evidence that is presented for OMPs to obtain marketing authorization in Europe, using the new and validated COMPASS tool.

Methods

We quantitatively assessed the characteristics and quality of clinical evidence of the pivotal studies of 64 OMPs as described in the European Public Assessment Report and/or the Scientific Discussion document prepared by the Committee for Human Medicinal Products of the EMA.

Results

The 64 OMPs were altogether authorized for 78 orphan indications, for which 117 studies were identified as 'pivotal’ or 'main’ studies. In approximately two thirds of the studies, the allocation was randomized (64.8%) and a control arm was used (68.5%). Half of the studies applied some type of blinding. Only a minority (26.9%) of the studies included a Quality-of-Life (QoL) related endpoint, of which a third claim an improvement in QoL. Upon analyzing the quality of reporting, we found that some aspects (i.e. the endpoints, the sampling criteria, and the interventions) are well described, whereas other items (i.e. a description of the patients and of potential biases) are not reported for all studies.

Conclusions

In conclusion, the pivotal studies that are the basis for marketing authorization of OMPs are a cause for concern, as they exhibit methodological flaws i.e. the lack of QoL-related endpoints as outcome, lack of blinding in the study design and the use of surrogate endpoints. Additionally, there are shortcomings in the reporting of those studies that complicate the interpretation. A more demanding regulatory process for OMPs is needed to guide evidence-based clinical decision-making.