Accumulating Evidence and Research Organization (AERO) model: a new tool for representing, analyzing, and planning a translational research program

Research encouraging off-label use of quetiapine: A systematic meta-epidemiological analysis

BACKGROUND

Researchers often conduct small studies on testing a drug's efficacy in off-label indications. If positive results from these exploratory studies are not followed up by larger, randomized, double-blinded trials, physicians cannot be sure of a drug's clinical value. This may lead to off-label prescriptions of ineffective treatments. We aim to describe the way clinical studies fostered off-label prescription of the antipsychotic drug quetiapine (Seroquel).

METHODS

In this systematic meta-epidemiological analysis, we searched EMBASE, MEDLINE, Cochrane CENTRAL and PsycINFO databases and included clinical studies testing quetiapine for unapproved indications between May 1995 and May 2022. We then assessed the frequency with which publications providing low-level evidence suggesting efficacy of quetiapine for off-label indications was not followed up by large, randomized and double-blinded trials within 5 years.

RESULTS

In total, 176 published studies were identified that reported potential efficacy of quetiapine in at least 26 indications. Between 2000 and 2007, publication of exploratory studies suggesting promise for off-label indications rapidly outpaced publication of confirmatory trials. In the 24 indications with a minimum of 5 years of follow-up from the first positive exploratory study, 19 (79%) were not followed up with large confirmatory trials within 5 years. At least nine clinical practice guidelines recommend the use of quetiapine for seven off-label indications in which published confirmatory evidence is lacking.

CONCLUSION

Many small, post-approval studies suggested the promise of quetiapine for numerous off-label indications. These findings generally went unconfirmed in large, blinded, randomized trials years after first being published. The imbalance of exploratory and confirmatory studies likely encourages ineffective off-label treatment.

Assessing patient burden and benefit: A decade of cabozantinib clinical trials

Drug development is complex and costly. Clinical trial participants take on risks, making it essential to maximize trial efficiency and maintain participant safety. Identifying periods of excessive burden during drug development can inform trial design, ensure patient benefit and prevent harm. This study aims to examine all published clinical trials for cabozantinib to assess patient benefit and burden over time. We conducted a retrospective cross-sectional review of interventional clinical trials of cabozantinib for solid cancer treatment. We searched PubMed/MEDLINE, Embase, Cochrane (CENTRAL) and ClinicalTrials.gov. We extracted adverse event rates, median progression-free survival (PFS), median overall survival and objective response rate (ORR) for each included trial. We calculated frequencies of trial characteristics, cumulative grade 3-5 adverse event rates and cumulative ORRs. Out of 1735 studies, 54 publications were included that involved 6372 participants and 21 cancers. Of the 54 studies in our sample, 31 (57.41%) were single-arm trials and 23 (42.60%) had negative results. Trials among and within various indications had conflicting results over time. Cumulative risk to participants increased over time, and clinical benefit decreased. The findings suggest that the risk profile of cabozantinib increased from 2011 to 2016 and has remained elevated but stable while benefit has decreased over time. The use of non-randomized and single-arm trials is concerning, and more methodologically rigorous trials are needed. The results of trials for different indications are inconsistent, and empirical administration may reduce the drug's efficacy.

Assessing Patient Risk, Benefit, and Outcomes in Drug Development: A Decade of Lenvatinib Clinical Trials: A Systematic Review

IMPORTANCE

Chemotherapy agents are typically initially tested in their most promising indications; however, following initial US FDA approval, new clinical trials are often initiated in less promising indications where patients experience a worse burden-benefit ratio. The current literature on the burden-benefit profile of lenvatinib in non-FDA-approved indications is lacking.

OBJECTIVE

This study aimed to evaluate published clinical trials of lenvatinib in order to determine the burden-benefit profile for patients over time.

EVIDENCE REVIEW

On 25 May 2023, we searched the Pubmed/MEDLINE, Embase, Cochrane CENTRAL, and ClinicalTrials.gov databases for clinical trials of lenvatinib used to treat solid cancers. Eligible articles were clinical trials, containing adult participants, published in English, and involving solid tumors. Screening and data collection took place in a masked, duplicate fashion. For each eligible study, we collected adverse event data, trial characteristics, progression-free survival (PFS), overall survival (OS), and objective response rate (ORR). Trials were classified as positive when meeting their primary endpoint and safety, negative (not meeting either criteria), or indeterminate (lacking prespecified primary endpoint).

FINDINGS

Expansion of clinical trial testing beyond lenvatinib's initial FDA indication demonstrated a consistent rise in cumulative adverse events, along with a decline in drug efficacy. Lenvatinib was tested in 16 cancer indications, receiving FDA approval in 4. A total of 5390 Grade 3-5 adverse events were experienced across 6225 clinical trial participants. Expanded indication testing further demonstrated widely variable ORR (11-69%), OS (6.2-32 months), and PFS (3.6-15.7 months) across all indications. After initial FDA approval, clinical trial results in expanded indications were less likely to meet their primary endpoints, particularly among non-randomized clinical trials.

CONCLUSION AND RELEVANCE

Our paper evaluated the effectiveness of lenvatinib for its FDA-approved indications; however, expansion of clinical trials into novel indications was characterized by diminished efficacy, while patients experienced a high burden of adverse events consistent with lenvatinib's established safety profile. Furthermore, clinical trials testing in novel indications was marked by repeated phase I and II clinical trials along with a failure to progress to phase III clinical trials. Future clinical trials using lenvatinib as an intervention should carefully evaluate the potential benefits and burden patients may experience.

Preclinical efficacy in investigator's brochures: Stakeholders' views on measures to improve completeness and robustness

AIMS

Research ethics committees and regulatory agencies assess whether the benefits of a proposed early-stage clinical trial outweigh the risks based on preclinical studies reported in investigator's brochures (IBs). Recent studies have indicated that the reporting of preclinical evidence presented in IBs does not enable proper risk-benefit assessment. We interviewed different stakeholders (regulators, research ethics committee members, preclinical and clinical researchers, ethicists, and metaresearchers) about their views on measures to increase the completeness and robustness of preclinical evidence reporting in IBs.

METHODS

This study was preregistered (https://osf.io/nvzwy/). We used purposive sampling and invited stakeholders to participate in an online semistructured interview between March and June 2021. Themes were derived using inductive content analysis. We used a strengths, weaknesses, opportunities and threats matrix to categorize our findings.

RESULTS

Twenty-seven international stakeholders participated. The interviewees pointed to several strengths and opportunities to improve completeness and robustness, mainly more transparent and systematic justifications for the included studies. However, weaknesses and threats were mentioned that could undermine efforts to enable a more thorough assessment: The interviewees stressed that current review practices are sufficient to ensure the safe conduct of first-in-human trials. They feared that changes to the IB structure or review process could overburden stakeholders and slow drug development.

CONCLUSION

In principle, more robust decision-making processes align with the interests of all stakeholders and with many current initiatives to increase the translatability of preclinical research and limit uninformative or ill-justified trials early in the development process. Further research should investigate measures that could be implemented to benefit all stakeholders.

The evidence landscape in precision medicine

Precision medicine is beginning to make an impact on the treatment of different diseases, but there are still challenges that must be overcome, such as the complexity of interventions, the need for marker validation, and the level of evidence necessary to demonstrate effectiveness. In this Perspective, we describe how evidence landscapes can help to address these challenges.

A Systematic Review and Meta-Analysis of Bevacizumab in First-Line Metastatic Breast Cancer: Lessons for Research and Regulatory Enterprises

BACKGROUND

The US Food and Drug Administration's accelerated approval and later withdrawal of bevacizumab in patients with metastatic breast cancer (mBC) is a seminal case for ongoing debates about the validity of using progression-free survival (PFS) as a surrogate measure for overall survival (OS) in cancer drug approvals. We systematically reviewed and meta-analyzed the evidence around bevacizumab's regulatory approval and withdrawal in mBC.

METHODS

We searched for all published phase II or III clinical trials testing bevacizumab as a first-line therapy for patients with mBC. Data were extracted on trial demographics, interventions, and outcomes. Descriptive analysis was stratified by whether the trial was initiated before, during, or after the accelerated approval. We used a cumulative random-effects meta-analysis to assess the evolution of evidence of the effect of bevacizumab on PFS and OS. We estimated the association between the trial-level PFS and OS effect using a nonlinear mixed-regression model.

RESULTS

Fifty-two studies were included. Trial activity dramatically dropped after the accelerated approval was withdrawn. Eight clinical trials reported hazard ratios (hazard ratios) and were meta-analyzed. The cumulative hazard ratio for PFS was 0.72 (95% CI = 0.65 to 0.79), and the cumulative hazard ratio for OS was 0.90 (95% CI = 0.80 to 1.01). The regression model showed a statistically nonsignificant association between PFS benefit and OS benefit (β = 0.43, SE = 0.81).

CONCLUSION

The US Food and Drug Administration's decision-making in this case was consistent with the evolving state of evidence. However, the fact that seven clinical trials are insufficient to conclude validity (or lack thereof) for a trial-level surrogate suggests that it would be more efficient to conduct trials using the more clinically meaningful endpoints.

From the Lab to Patients: a Systematic Review and Meta-Analysis of Mesenchymal Stem Cell Therapy for Stroke

There may be the potential to improve stroke recovery with mesenchymal stem cells (MSCs); however, questions about the efficacy and safety of this treatment remain. To address these issues and inform future studies, we performed a preclinical and clinical systematic review of MSC therapy for subacute and chronic ischemic stroke. MEDLINE, Embase, the Cochrane Register of Controlled Trials, and PubMed were searched. For the clinical review, interventional and observational studies of MSC therapy in ischemic stroke patients were included. For the preclinical review, interventional studies of MSC therapy using in vivo animal models of subacute or chronic stroke were included. Measures of safety and efficacy were assessed. Eleven clinical and 76 preclinical studies were included. Preclinically, MSC therapy was associated with significant benefits for multiple measures of motor and neurological function. Clinically, MSC therapy appeared to be safe, with no increase in adverse events reported (with the exception of self-limited fever immediately following injection). However, the efficacy of treatment was less apparent, with significant heterogeneity in both study design and effect size being observed. Additionally, in the only randomized phase II study to date, efficacy of MSC therapy was not observed. Preclinically, MSC therapy demonstrated considerable efficacy. Although MSC therapy demonstrated safety in the clinical setting, efficacy has yet to be determined. Future studies will need to address the discordance in the continuity of evidence as MSC therapy has been translated from "bench-to-bedside".

Assessment of Pregabalin Postapproval Trials and the Suggestion of Efficacy for New Indications: A Systematic Review

IMPORTANCE

After a drug receives regulatory approval, researchers often pursue small, underpowered trials, called exploratory trials, aimed at testing additional indications. If favorable early findings from exploratory trials are not promptly followed by confirmatory trials, then physicians, patients, and payers can be left uncertain about a drug's clinical value (clinical agnosticism). Such findings may encourage the off-label use of ineffective drugs.

OBJECTIVE

To characterize the relationship between exploratory and confirmatory postapproval trials for the blockbuster drug, pregabalin (Lyrica).

EVIDENCE REVIEW

Ovid MEDLINE and Embase databases were used to identify clinical trials published prior to January 2018 and that tested the efficacy of pregabalin for nonapproved indications. Indications, trial outcomes, publication dates, and trial design elements were recorded. Time elapsed was calculated between the generation of clinical agnosticism about pregabalin (ie, publications reporting positive or inconclusive evidence of efficacy on a primary endpoint) and it being addressed (publication of at least 1 confirmatory trial in the same indication, regardless of outcome).

FINDINGS

There were 238 trials identified that tested the efficacy of pregabalin in at least 33 indications; 5 indications eventually received European Medicines Agency and/or US Food and Drug Administration marketing approval. Sixty-seven percent (22 of 33) of first publications for new indications may have generated clinical agnosticism. Of those indications with at least 5 years of follow-up, 63% (17 of 27) may have generated agnosticism that was not addressed in confirmatory trials within 5 years. As pregabalin development expanded from indications that received regulatory approval to other indications, the linkage of exploratory to confirmatory trial publication diminished.

CONCLUSIONS AND RELEVANCE

After initial approval, exploratory evidence suggesting the value of pregabalin for new indications often went unconfirmed for extended periods of time. Poor coordination between exploratory and confirmatory testing may represent an important vehicle through which off-label prescription is recommended in clinical practice guidelines and encouraged in the absence of confirmatory trial evidence.

Reprioritizing Research Activity for the Post-Antibiotic Era: Ethical, Legal, and Social Considerations

Many hold that the so-called golden era of antibiotic discovery has passed, leaving only a limited clinical pipeline for new antibiotics. A logical conclusion of such arguments is that we need to reform the current system of antibiotic drug research-including clinical trials and regulatory requirements-to spur activity in discovery and development. The United States Congress in the past few years has debated a number of bills to address this crisis, including the 2012 Generating Antibiotic Incentives Now Act and the 2016 21st Century Cures Act. Experts have also sought to advance antibiotic development by encouraging greater use of trials with noninferiority hypotheses, which are thought to be easier to conduct. The goal underlying these proposals is to stave off the post-antibiotic era by expanding the pharmaceutical armamentarium as quickly as possible. But although new antibiotic agents are necessary to combat the long-term threat of drug-resistant disease, we argue that these research policies, which effectively lower the bar for antibiotic approval, are ethically problematic. Rather, given broader public health considerations related to the full lifecycle of antibiotic use-including development of resistance-we should reject an overly permissive approach to new antibiotic approval and instead set the bar for regulatory approval at a point that will naturally direct research resources toward the most transformative chemical or social interventions.

The Challenges of Validating in Precision Medicine: The Case of Excision Repair Cross-Complement Group 1 Diagnostic Testing

Personalized medicine relies upon the successful identification and translation of predictive biomarkers. Unfortunately, biomarker development has often fallen short of expectations. To better understand the obstacles to successful biomarker development, we systematically mapped research activities for a biomarker that has been in development for at least 12 years: excision repair cross-complement group 1 protein (ERCC1) as a biomarker for predicting clinical benefit with platinum-based chemotherapy in non-small cell lung cancer. We found that although research activities explored a wide range of approaches to ERCC1 testing, there was little replication or validation of techniques, and design and reporting of results were generally poor. Our analysis points to problems with coordinating and standardizing research in biomarker development. Clinically meaningful progress in personalized medicine will require concerted efforts to address these problems. In the interim, health care providers should be aware of the complexity involved in biomarker development, cautious about their near-term clinical value, and conscious of applying only validated diagnostics in the clinic.

THE ONCOLOGIST

2017;22:89-96 IMPLICATIONS FOR PRACTICE: : Many hospitals, policy makers, and scientists have made ambitious claims about the promise of personalizing cancer care. When one uses a case example of excision repair cross-complement group 1 protein-a biomarker that has a strong biological rationale and that has been researched for 12 years-the current research environment seems poorly suited for efficient development of biomarker tests. The findings provide grounds for tempering expectations about personalized cancer care-at least in the near term-and shed light on the current gap between the promise and practice of personalized medicine.

Benefit, Risk, and Outcomes in Drug Development: A Systematic Review of Sunitinib

BACKGROUND

Little is known about the total patient burden associated with clinical development and where burdens fall most heavily during a drug development program. Our goal was to quantify the total patient burden/benefit in developing a new drug.

METHODS

We measured risk using drug-related adverse events that were grade 3 or higher, benefit by objective response rate, and trial outcomes by whether studies met their primary endpoint with acceptable safety. The differences in risk (death rate) and benefit (overall response rate) between industry and nonindustry trials were analyzed with an inverse-variance weighted fixed effects meta-analysis implemented as a weighted regression analysis. All statistical tests were two-sided.

RESULTS

We identified 103 primary publications of sunitinib monotherapy, representing 9092 patients and 3991 patient-years of involvement over 10 years and 32 different malignancies. In total, 1052 patients receiving sunitinib monotherapy experienced objective tumor response (15.7% of intent-to-treat population, 95% confidence interval [CI] = 15.3% to 16.0%), 98 died from drug-related toxicities (1.08%, 95% CI = 1.02% to 1.14%), and at least 1245 experienced grade 3-4 drug-related toxicities (13.7%, 95% CI = 13.3% to 14.1%). Risk/benefit worsened as the development program matured, with several instances of replicated negative studies and almost no positive trials after the first responding malignancies were discovered.

CONCLUSIONS

Even for a successful drug, the risk/benefit balance of trials was similar to phase I cancer trials in general. Sunitinib monotherapy development showed worsening risk/benefit, and the testing of new indications responded slowly to evidence that sunitinib monotherapy would not extend to new malignancies. Research decision-making should draw on evidence from whole research programs rather than a narrow band of studies in the same indication.

Ethics and epistemology of accurate prediction in clinical research

All major research ethics policies assert that the ethical review of clinical trial protocols should include a systematic assessment of risks and benefits. But despite this policy, protocols do not typically contain explicit probability statements about the likely risks or benefits involved in the proposed research. In this essay, I articulate a range of ethical and epistemic advantages that explicit forecasting would offer to the health research enterprise. I then consider how some particular confidence levels may come into conflict with the principles of ethical research.