Quality and quantity of information in summary basis of decision documents issued by health Canada

Quality and quantity of data used by Health Canada in approving new drugs

Background

This study examined multiple aspects about the approval of new drugs: the characteristics of the drugs, the quality and quantity of information that Health Canada discloses about the demographics of patients enrolled in clinical trials, the characteristics of the trial, and the type of review that it uses. It examines whether there have been changes in these measures between 1 September 2012 and 31 March 2022.

Methods

A list of all new drugs approved, type of review used, and drug characteristics was generated from Health Canada annual reports. Therapeutic categories were identified from the World Health Organization Collaborating Center for Drugs Statistics Methodology. The Summary Basis of Decision documents of Health Canada were used to identify patient demographics in clinical trials and clinical trial characteristics.

Results

Health Canada approved 326 new drugs for 407 indications. The percent of orphan drugs approved increased from 35.6 to 51.3%. The number of indications per drug decreased (p = 0.0817) as did the number of pivotal trials per drug (p = 0.0091). The percent of Phase 3 trials dropped from 76.3% in 2012-2015 to 64.8% in 2019-2022 (p = 0.005). There was also a statistically significant decrease in the percent of trials that were randomized, controlled, and blinded. The clinical trial characteristics of orphan drugs and the type of review used were both significantly different compared with non-orphan drugs. The percent of trials which had information about the number of patients enrolled, the percent of trials that provided the age of the patients, and the sex breakdown all significantly increased.

Conclusion

The results show that there has been a change in regulatory standards that may be due to them becoming less rigorous, because of an adaptation to the number of orphan drugs being submitted or a combination of both reasons. At the same time, there has been some improvement in the transparency of data. Health Canada has recently embarked on a series of reforms in drug regulation and clinical trial management. These changes need to be closely evaluated to be sure that they enhance the efficacy and safety of new drugs.

Information about confirmatory studies required for new drugs conditionally approved by Health Canada: A cross-sectional study

BACKGROUND

Health Canada conditionally approves new drugs using its Notice of Compliance with conditions (NOC/c) policy. Under this policy Qualifying Notices (QNs) list confirmatory studies that need to be conducted to confirm the drug's efficacy. This study examines the depth of information about methodology and patient demographics in the confirmatory studies. It also compares the outcomes (surrogate or clinical) used to approve the drugs with the outcomes proposed in the confirmatory studies.

METHODS

A list of drugs approved under the NOC/c policy and their QNs were sourced from two previous publications as well as Health Canada's NOC/c website. Patient demographics and study methodology in the confirmatory studies listed in the QNs was recorded and counted. The primary outcome used to approve new drugs was recorded from Health Canada's Summary Basis of Decision website and compared to the type of outcome for studies mentioned in the QNs.

RESULTS

Seventy-eight drugs were approved using a NOC/c from the time the first drug was approved under the program in July 1998 until May 18, 2022. QNs were missing or all information was redacted for 3 drugs, the remaining 75 QNs listed 154 studies (median of 2 studies per QN, interquartile range 1,3). The outcome, randomization and blinding could not be determined for any study in 43 (57.3%), 36 (48.0%) and 42 (56.0%) QNs, respectively. No study gave the distribution of men and women and the number of patients was given in 23 (14.9%) studies. The expected time of completion of the studies was available for 36 (23.4%) out of 154 and information to identify studies was present for 77 (50.0%), absent for 23 (14.9%) and unclear for 26 (16.9%). Surrogate outcomes were used to approve 54 (84.4%) of 64 drugs. Eight (14.8%) confirmatory studies for these 54 drugs used clinical outcomes, 15 (27.8%) used surrogate outcomes and outcomes were unknown for 31 (57.4%). Specifically for oncology drugs, 44 were approved with surrogate outcomes and one with a clinical outcome. Eight (18.2%) of the 44 oncology drugs approved with surrogate outcomes had confirmatory studies that used clinical outcomes, 14 (31.8%) used surrogate outcomes and the outcome could not be determined for 22 (50.0%). The sole oncology drug approved with a clinical outcome had a confirmatory study with a surrogate outcome.

DISCUSSION

QNs contain little information about the methodology or patient demographics of confirmatory studies. Confirmatory studies with surrogate outcomes were used almost one-third of the time to validate efficacy in drugs initially approved using surrogate outcomes. Health Canada needs to develop a template about what information regarding confirmatory studies should be contained in a QN and rethink its use of confirmatory studies using surrogate outcomes. All the data were gathered by a single individual possibly introducing unintended biases.

Quality of evidence considered by Health Canada in granting full market authorisation to new drugs with a conditional approval: a retrospective cohort study

OBJECTIVES

This study examines the characteristics of studies that Health Canada uses to grant full marketing authorisation for products given a conditional approval between 1 January 1998 and 30 June 2017.

DESIGN

Cohort study.

DATA SOURCES

Journal articles listing drugs that fulfilled their conditions and received full marketing authorisation, Notice of Compliance database, Notice of Compliance with conditions website, Qualifying Notices listing required confirmatory studies, clinicaltrials.gov, PubMed, Embase, companies making products being analysed, journal articles resulting from confirmatory studies.

INTERVENTIONS

None.

PRIMARY AND SECONDARY OUTCOME MEASURES

Characteristics of studies-study design (randomised controlled trials, observational), primary outcome used (clinical, surrogate), blinding, number of patients in studies, patient median age, number of men and women.

RESULTS

Eleven companies confirmed 36 publications for 19 products (21 indications). Twenty-nine out of the 36 studies were randomised controlled trials (RCTs) but only 10 stated if they were blinded. Twenty used surrogate outcomes. The median age of patients was 56 (IQR 44-61). The median number of men per study/trial was 184 (IQR 58-514) versus women 141 (IQR 46-263).

CONCLUSIONS

Postmarket studies required by Health Canada had more rigorous methodology than those required by either the Food and Drug Administration or the European Medicines Agency. There were still deficiencies in these studies. The absence of blinding in the majority of RCTs may introduce bias in their results. The use of surrogate outcomes especially in oncology trials means that improvements in survival are not available. The relatively young age of patients, even for products for cancer, means that predicting how the elderly will respond is often unknown. The almost universal finding that men outnumbered women may make it hard to differentiate responses by sex. These results raise potential concerns about the quality of evidence that Health Canada accepts.

Postmarket safety of drugs approved by Health Canada on the basis of clinical and surrogate outcomes: a cohort study

BACKGROUND

Health Canada approves drugs on the basis of evidence from clinical trials using clinical or surrogate outcomes. This study compares the postmarket safety of these 2 groups of drugs.

METHODS

Information about whether clinical or surrogate outcomes were used and the date of market approval were obtained from Health Canada's Summary Basis of Decision documents issued from Jan. 1, 2005, to Dec. 31, 2014. Safety warnings and the dates they were issued were identified through advisories on the MedEffect Canada website. Kaplan-Meier survival curves were calculated to determine the likelihood that drugs in the clinical and surrogate outcome groups would receive a serious safety warning. The time from market authorization to first serious safety warning was compared for the 2 groups of drugs.

RESULTS

A total of 124 drugs were approved by Health Canada using clinical outcomes and 114 using surrogate outcomes. Kaplan-Meier curves did not differ between the 2 groups (p < 0.9). The median time from market authorization to first serious safety warning was 722 days in the clinical outcome group and 818 days in the surrogate outcome group (difference 96 days, 95% confidence interval -295 to 425).

INTERPRETATION

We found no statistically significant difference in postmarket safety between drugs approved using clinical outcomes and those approved using surrogate outcomes. Because drugs in the surrogate outcome group are approved before their benefit:harm ratio is fully established, these drugs should be used with caution until their clinical benefits are better understood.

Information on new drugs at market entry: retrospective analysis of health technology assessment reports versus regulatory reports, journal publications, and registry reports

BACKGROUND

When a new drug becomes available, patients and doctors require information on its benefits and harms. In 2011, Germany introduced the early benefit assessment of new drugs through the act on the reform of the market for medicinal products (AMNOG). At market entry, the pharmaceutical company responsible must submit a standardised dossier containing all available evidence of the drug's added benefit over an appropriate comparator treatment. The added benefit is mainly determined using patient relevant outcomes. The "dossier assessment" is generally performed by the Institute for Quality and Efficiency in Health Care (IQWiG) and then published online. It contains all relevant study information, including data from unpublished clinical study reports contained in the dossiers. The dossier assessment refers to the patient population for which the new drug is approved according to the summary of product characteristics. This patient population may comprise either the total populations investigated in the studies submitted to regulatory authorities in the drug approval process, or the specific subpopulations defined in the summary of product characteristics ("approved subpopulations").

OBJECTIVE

To determine the information gain from AMNOG documents compared with non-AMNOG documents for methods and results of studies available at market entry of new drugs. AMNOG documents comprise dossier assessments done by IQWiG and publicly available modules of company dossiers; non-AMNOG documents comprise conventional, publicly available sources-that is, European public assessment reports, journal publications, and registry reports. The analysis focused on the approved patient populations.

DESIGN

Retrospective analysis.

DATA SOURCES

All dossier assessments conducted by IQWiG between 1 January 2011 and 28 February 2013 in which the dossiers contained suitable studies allowing for a full early benefit assessment. We also considered all European public assessment reports, journal publications, and registry reports referring to these studies and included in the dossiers.

DATA ANALYSIS

We assessed reporting quality for each study and each available document for eight methods and 11 results items (three baseline characteristics and eight patient relevant outcomes), and dichotomised them as "completely reported" or "incompletely reported (including items not reported at all)." For each document type we calculated the proportion of items with complete reporting for methods and results, for each item and overall, and compared the findings.Results 15 out of 27 dossiers were eligible for inclusion and contained 22 studies. The 15 dossier assessments contained 28 individual assessments of 15 total study populations and 13 approved subpopulations. European public assessment reports were available for all drugs. Journal publications were available for 14 out of 15 drugs and 21 out of 22 studies. A registry report in ClinicalTrials.gov was available for all drugs and studies; however, only 11 contained results. In the analysis of total study populations, the AMNOG documents reached the highest grade of completeness, with about 90% of methods and results items completely reported. In non-AMNOG documents, the rate was 75% for methods and 52% for results items; journal publications achieved the best rates, followed by European public assessment reports and registry reports. The analysis of approved subpopulations showed poorer complete reporting of results items, particularly in non-AMNOG documents (non-AMNOG versus AMNOG: 11% v 71% for overall results items and 5% v 70% for patient relevant outcomes). The main limitation of our analysis is the small sample size.

CONCLUSION

Conventional, publicly available sources provide insufficient information on new drugs, especially on patient relevant outcomes in approved subpopulations. This type of information is largely available in AMNOG documents, albeit only partly in English. The AMNOG approach could be used internationally to develop a comprehensive publication model for clinical studies and thus represents a key open access measure.

Why are there deadly drugs?

Some drugs eventually have to be removed from the market because of a negative benefit-to-harm ratio, including an excess of mortality. Drug safety is the result of multiple factors, commencing with how clinical trials are designed, the information generated by and/or hidden through these trials, trial analysis by drug regulatory authorities (DRAs) and the amount of information that DRAs choose to release, the amount of published information regarding drug safety, the effectiveness of postmarket surveillance systems in recognizing and reporting adverse drug reactions, and the structure of DRAs such as the United States Food and Drug Administration and its equivalent in other countries. This commentary will look at each of these issues in order to highlight the problems in the current approach to drug safety and finally indicate how some of these deficiencies should be addressed.

Clinical trial transparency: many gains but access to evidence for new medicines remains imperfect

BACKGROUND

Although selective and incomplete publication is widely acknowledged to be a problem, full access to clinical trial data remains illusive.

SOURCES OF DATA

Authors' personal files, key documents from Food and Drug Administration and European Medicines Agency and focussed searches of PubMed.

AREAS OF AGREEMENT

Existing sources of information provide an incomplete overview of scientific research.

AREAS OF CONTROVERSY

Persistent arguments about commercial confidentiality and the potential difficulties in de-identifying raw data can block important progress. Current industry efforts are voluntary and only partially satisfy the need for complete data.

GROWING POINTS

Requirements for trial registration are increasing. Important regulatory changes in particular in Europe have the potential to result in the release of more information.

AREAS TIMELY FOR DEVELOPING RESEARCH

Documenting the effects of prospective trial registration and requirements for proactive clinical trial publication on healthcare decisions, public health and rational resource allocation.