Use of surrogate outcomes in US FDA drug approvals, 2003-2012: a survey

Trends in the Quality of Evidence Supporting FDA Drug Approvals: Results from a Literature Review

CONTEXT

New drug approvals in the United States must be supported by substantial evidence from "adequate and well-controlled" trials. The Food and Drug Administration (FDA) has flexibility in how it applies this standard.

METHODS

The authors conducted a systematic literature review of studies evaluating the design and outcomes of the key trials supporting new drug approvals in the United States. They extracted data on the trial characteristics, endpoint types, and expedited regulatory pathways.

FINDINGS

Among 48 publications eligible for inclusion, 30 covered trial characteristics, 23 covered surrogate measures, and 30 covered regulatory pathways. Trends point toward less frequent randomization, double-blinding, and active controls, with variation by drug type and indication. Surrogate measures are becoming more common but are not consistently well correlated with clinical outcomes. Drugs approved through expedited regulatory pathways often have less rigorous trial design characteristics.

CONCLUSIONS

The characteristics of trials used to approve new drugs have evolved over the past two decades along with greater use of expedited regulatory pathways and changes in the nature of drugs being evaluated. While flexibility in regulatory standards is important, policy changes can emphasize high-quality data collection before or after FDA approval.

Is HbA1c a valid surrogate for mortality in type 2 diabetes? Evidence from a meta-analysis of randomized trials

AIMS

Hemoglobin A1c (HbA1c) has been repeatedly questioned as a valid surrogate marker, especially for patient-relevant outcomes. The aim of this study was to validate the HbA1c value as a surrogate for all-cause mortality in people with type 2 diabetes.

METHODS

The effect estimates for HbA1c lowering after treatment as well as reductions in all-cause mortality of randomized trials were extracted from a systematic review and updated. For the measurement of actual surrogacy, weighted linear regression models with a random intercept for the study effect were used with the all-cause mortality estimate (risk difference and log relative risk) as the outcome and the estimate for HbA1c difference as the covariate. Surrogacy was assessed according to the criteria of Daniels and Hughes.

RESULTS

A total of 346 HbA1c-mortality-pairs from 205 single randomized trials were included in the analysis. Regarding the risk difference of all-cause mortality, there was no evidence for surrogacy of the HbA1c value. For the log relative risk, a small positive association between HbA1c and the all-cause mortality estimate (slope 0.129 [95% confidence interval -0.043; 0.302]) was observed. However, there was no sign of valid surrogacy.

CONCLUSIONS

Based on the results of more than 200 randomized trials, HbA1c is not a valid surrogate marker for all-cause mortality in people with type 2 diabetes.

Treatment-Related Changes in Bone Turnover and Fracture Risk Reduction in Clinical Trials of Antiresorptive Drugs: Proportion of Treatment Effect Explained

Few analyses of antiresorptive (AR) treatment trials relate short-term changes in bone turnover markers (BTMs) to subsequent fracture reduction seeking to estimate the proportion of treatment effect explained (PTE) by BTMs. Pooling such information would be useful to assess new ARs or novel dosing regimens. In the Foundation for the National Institutes of Health (FNIH) Bone Quality project, we analyzed individual-level data from up to 62,000 participants enrolled in 12 bisphosphonate (BP) and four selective estrogen receptor modulator (SERM) placebo-controlled fracture endpoint trials. Using BTM results for two bone formation markers (bone-specific alkaline phosphatase [bone ALP] and pro-collagen I N-propeptide [PINP]) and one bone resorption marker (C-terminal telopeptide of type I collagen [CTX]) and incident fracture outcome data, we estimated the PTE using two different models. Separate analyses were performed for incident morphometric vertebral, nonvertebral, and hip fractures over 1 to 5 years of follow-up. For vertebral fracture, the results showed that changes in all three BTMs at 6 months explained a large proportion of the treatment effect of ARs (57 to >100%), but not for and non-vertebral or hip fracture. We conclude that short-term AR treatment-related changes in bone ALP, PINP, and CTX account for a large proportion of the treatment effect for vertebral fracture. Change in BTMs is a useful surrogate marker to study the anti-fracture efficacy of new AR compounds or novel dosing regiments with approved AR drugs. © 2020 The Authors. Journal of Bone and Mineral Research published by American Society for Bone and Mineral Research.

Informing Patient-Centered Care Through Stakeholder Engagement and Highly Stratified Quantitative Benefit-Harm Assessments

OBJECTIVES

In a previous project aimed at informing patient-centered care for people with multiple chronic conditions, we performed highly stratified quantitative benefit-harm assessments for 2 top priority questions. In this current work, our goal was to describe the process and approaches we developed and to qualitatively glean important elements from it that address patient-centered care.

METHODS

We engaged patients, caregivers, clinicians, and guideline developers as stakeholder representatives throughout the process of the quantitative benefit-harm assessment and investigated whether the benefit-harm balance differed based on patient preferences and characteristics (stratification). We refined strategies to select the most applicable, valid, and precise evidence.

RESULTS

Two processes were important when assessing the balance of benefits and harms of interventions: (1) engaging stakeholders and (2) stratification by patient preferences and characteristics. Engaging patients and caregivers through focus groups, preference surveys, and as co-investigators provided value in prioritizing research questions, identifying relevant clinical outcomes, and clarifying the relative importance of these outcomes. Our strategies to select evidence for stratified benefit-harm assessments considered consistency across outcomes and subgroups. By quantitatively estimating the range in the benefit-harm balance resulting from true variation in preferences, we clarified whether the benefit-harm balance is preference sensitive.

CONCLUSIONS

Our approaches for engaging patients and caregivers at all phases of the stratified quantitative benefit-harm assessments were feasible and revealed how sensitive the benefit-harm balance is to patient characteristics and individual preferences. Accordingly, this sensitivity can suggest to guideline developers when to tailor recommendations for specific patient subgroups or when to explicitly leave decision making to individual patients and their providers.

Prioritizing outcome preferences in patients with ocular hypertension and open-angle glaucoma using best-worst scaling

Purpose

To quantify patients' preferences for glaucoma outcomes and use this information to prioritize outcomes that are important to patients.

Design

A cross-sectional study using best-worst scaling object case (BWS).

Participants

Two hundred seventy-four participants newly diagnosed with ocular hypertension or mild to moderate open angle glaucoma from three private practices and one academic medical center in the United States.

Methods

We designed a preference-elicitation survey based on findings from 25 semi-structured, qualitative interviews with patients with glaucoma (reported elsewhere). The survey asked participants to rate the importance of 13 glaucoma outcomes on a Likert scale as a warm-up exercise followed by completion of 13 BWS tasks. For each task, we presented participants a subset of four outcomes from the possible thirteen, and participants chose the most important and least important outcome. Outcomes included in the survey pertain to maintaining ability to perform vision-dependent activities of daily living (e.g., driving), maintaining visual function and perception (e.g., depth perception), minimizing need to take glaucoma drops, not experiencing ocular surface symptoms (e.g., red eyes, teary eyes), and having adequate control of intraocular pressure (IOP). We administered the survey online and analyzed response patterns using conditional logistic regression to determine the relative importance of different outcomes.

Main outcome

Ordinal ranking of glaucoma outcomes based on preference weights.

Results

Between September 1, 2017 and February 28, 2018, we invited 1035 patients to complete our survey, among whom 274 (26%) responded. Most participants were older than 65 years of age (146/274, 53%) and currently on IOP-lowering drops (179/274, 65%). Participants identified that outcomes with the largest relative importance weight were having "adequate IOP control" and ability to "drive a car during the day," and the outcomes with the smallest relative importance weights were "maintaining appearance of the eye" and "reducing the number of IOP-lowering drops".

Conclusions

Determining the relative importance of glaucoma outcomes to patients can help researchers design studies that may better inform clinical and regulatory decision-making. Although IOP is an outcome that researchers often measure in glaucoma clinical trials, patients also prioritized outcomes related to the ability to perform vision-dependent activities such as driving.

Clinical drug trials in general practice: how well are external validity issues reported?

BACKGROUND

When reading a report of a clinical trial, it should be possible to judge whether the results are relevant for your patients. Issues affecting the external validity or generalizability of a trial should therefore be reported. Our aim was to determine whether articles with published results from a complete cohort of drug trials conducted entirely or partly in general practice reported sufficient information about the trials to consider the external validity.

METHODS

A cohort of 196 drug trials in Norwegian general practice was previously identified from the Norwegian Medicines Agency archive with year of application for approval 1998-2007. After comprehensive literature searches, 134 journal articles reporting results published from 2000 to 2015 were identified. In these articles, we considered the reporting of the following issues relevant for external validity: reporting of the clinical setting; selection of patients before inclusion in a trial; reporting of patients' co-morbidity, co-medication or ethnicity; choice of primary outcome; and reporting of adverse events.

RESULTS

Of these 134 articles, only 30 (22%) reported the clinical setting of the trial. The number of patients screened before enrolment was reported in 61 articles (46%). The primary outcome of the trial was a surrogate outcome for 60 trials (45%), a clinical outcome for 39 (29%) and a patient-reported outcome for 25 (19%). Clinical details of adverse events were reported in 124 (93%) articles. Co-morbidity of included participants was reported in 54 trials (40%), co-medication in 27 (20%) and race/ethnicity in 78 (58%).

CONCLUSIONS

The clinical setting of the trials, the selection of patients before enrolment, and co-morbidity or co-medication of participants was most commonly not reported, limiting the possibility to consider the generalizability of a trial. It may therefore be difficult for readers to judge whether drug trial results are applicable to clinical decision-making in general practice or when developing clinical guidelines.

Timing and Characteristics of Cumulative Evidence Available on Novel Therapeutic Agents Receiving Food and Drug Administration Accelerated Approval

Policy Points: Randomized trials-the gold standard of evaluating effectiveness-constitute a small minority of existing evidence on agents given accelerated approval. One-third of randomized trials are in therapeutic areas outside of FDA approval and less than half evaluate the therapeutic benefits of these agents but use them instead as common backbone treatments. Agents receiving accelerated approval are often tested concurrently in several therapeutic areas. For most agents, no substantial time lag is apparent between the average start dates of randomized trials evaluating their effectiveness and those using them as part of background therapies. There appears to be a tendency for therapeutic agents receiving accelerated approval to quickly become an integral component of standard treatment, despite potential shortcomings in their evidence base.

CONTEXT

Therapeutic agents treating serious conditions are eligible for Food and Drug Administration (FDA) accelerated approval. The clinical evidence accrued on agents receiving accelerated approval has not been systematically evaluated. Our objective was to assess the timing and characteristics of available studies.

METHODS

We first identified clinical studies of novel therapeutic agents receiving accelerated approval. We then (1) categorized those studies as randomized or nonrandomized, (2) explored whether they evaluated the FDA-approved indications, and (3) documented the available treatment comparisons. We also meta-analyzed the difference in start times between randomized studies that (1) did or did not evaluate approved indications and (2) were or were not designed to evaluate the agent's effectiveness.

FINDINGS

In total, 37 novel therapeutic agents received accelerated approval between 2000 and 2013. Our search of ClinicalTrials.gov identified 7,757 studies, which included 1,258,315 participants. Only one-third of identified studies were randomized controlled trials. Of 1,631 randomized trials with advanced recruitment status, 906 were conducted in therapeutic areas for which agents received initial accelerated approval, 202 were in supplemental indications, and 523 were outside approved indications. Only 411 out of 906 (45.4%) trials were designed to test the effectiveness of agents that received accelerated approval ("evaluation" trials); others used these agents as common background treatment in both arms ("background" trials). There was no detectable lag between average start times of trials conducted within and outside initially approved indications. Evaluation trials started on average 1.52 years (95% CI: 0.87 to 2.17) earlier than background trials.

CONCLUSIONS

Cumulative evidence on agents with accelerated approvals has major limitations. Most clinical studies including these agents are small and nonrandomized, and about a third are conducted in unapproved areas, typically concurrently with those conducted in approved areas. Most randomized trials including these therapeutic agents are not designed to directly evaluate their clinical benefits but to incorporate them as standard treatment.

Differences in Treatment Effect Size Between Overall Survival and Progression-Free Survival in Immunotherapy Trials: A Meta-Epidemiologic Study of Trials With Results Posted at ClinicalTrials.gov

Purpose

We aimed to compare treatment effect sizes between overall survival (OS) and progression-free survival (PFS) in trials of US Food and Drug Administration–approved oncology immunotherapy drugs with results posted at ClinicalTrials.gov .

Methods

We searched ClinicalTrials.gov for phase II to IV cancer trials of Food and Drug Administration–approved immunotherapy drugs and selected those reporting results for both OS and PFS. For each trial, we extracted the hazard ratios (HRs) with 95% CIs for both outcomes and evaluated the differences by a ratio of HRs (rHRs): the HR for PFS to that for OS. We performed a random effects meta-analysis across trials to obtain a summary rHR. We also evaluated surrogacy of PFS for OS by the coefficient of determination and the surrogacy threshold effect, the minimal value of HR for PFS to predict a non-null effect on OS.

Results

We identified 51 trials assessing 14 drugs across 15 conditions. Treatment effect sizes were 17% greater, on average, with PFS than with OS (rHR, 0.83; 95% CI, 0.79 to 0.88; I2 = 34.4%; P = .01; τ2 = 0.0129). Nearly one half of the trials (n = 23, 45%) showed statistically significant benefits for PFS but not for OS. Differences were great for trials of obinutuzumab (rHR, 0.21; 95% CI, 0.08 to 0.54), bevacizumab (rHR, 0.75; 95% CI, 0.67 to 0.84), and rituximab (rHR, 0.79; 95% CI, 0.64 to 0.98). The coefficient of determination was 38% and the surrogacy threshold effect was 0.50.

Conclusion

Treatment effect sizes in trials of immunotherapy drugs were greater for PFS than for OS, with important differences for some drugs, which is consistent with surrogacy metrics. Caution must be taken when interpreting PFS in the absence of OS data.

The Methodology of Clinical Studies Used by the FDA for Approval of High-Risk Orthopaedic Devices

BACKGROUND

The purpose of this investigation was to examine the methodology of clinical trials used by the U.S. Food and Drug Administration (FDA) to determine the safety and effectiveness of high-risk orthopaedic devices approved between 2001 and 2015.

METHODS

Utilizing the FDA's online public database, this systematic review audited study design and methodological variables intended to minimize bias and confounding. An additional analysis of blinding as well as the Checklist to Evaluate a Report of a Nonpharmacological Trial (CLEAR NPT) was applied to the randomized controlled trials (RCTs).

RESULTS

Of the 49 studies, 46 (94%) were prospective and 37 (76%) were randomized. Forty-seven (96%) of the studies were controlled in some form. Of 35 studies that reported it, blinding was utilized in 21 (60%), of which 8 (38%) were reported as single-blinded and 13 (62%) were reported as double-blinded. Of the 37 RCTs, outcome assessors were clearly blinded in 6 (16%), whereas 15 (41%) were deemed impossible to blind as implants could be readily discerned on imaging. When the CLEAR NPT was applied to the 37 RCTs, >70% of studies were deemed "unclear" in describing generation of allocation sequences, treatment allocation concealment, and adequate blinding of participants and outcome assessors.

CONCLUSIONS

This study manifests the highly variable reporting and strength of clinical research methodology accepted by the FDA to approve high-risk orthopaedic devices.