Quality of reporting of serious adverse drug events to an institutional review board: a case study with the novel cancer agent, imatinib mesylate

Systemic Barriers and Potential Concerns from Reporting Serious Adverse Drug Reactions

About 1-10% of all serious adverse drug reactions (sADRs) are reported to the Food and Drug Administration (FDA) ( Moore T, Bennett C. Underreporting of Hemorrhagic and Thrombotic Complications of Pharmaceuticals to the U.S. Food and Drug Administration: Empirical Findings for Warfarin, Clopidogrel, Ticlopidine, and Thalidomide from the Southern Network on Adverse Reactions (SONAR). Semin Thromb Hemost. 2012;38(08):905-907. https://doi.org/10.1055/s-0032-1328890 ). Prevailing opinion suggests that low reporting rates reflect time constraints.

Effect of adalimumab interventions on general infection among adults: a systematic review and meta-analysis of randomized controlled trials

BACKGROUND

This study assessed the safety of adalimumab in different dosages and durations of treatment.

METHODS

We conducted a systematic review and meta-analysis of randomized controlled trials (RCTs) to explore the infection risk in people who received adalimumab. We searched the Cochrane Library, PubMed, and EMBASE from inception to December 8, 2020. Summary estimates were obtained  using meta-analysis with a random-effects model.

RESULTS

Twenty-one RCTs, considered to be of high quality, were analyzed. We found that there was a risk of infection (RR: 1.10, 95% CI: 1.02-1.18). In the stratified analysis, we found an increase in infection among those that received normal dosage (RR: 1.13, 95% CI: 1.04-1.23), and in patients with psoriasis (RR: 1.13, 95% CI: 1.00-1.35) and rheumatoid arthritis (RR: 1.23, 95% CI: 1.06-1.41), but not in those that received high doses and other criteria. In the meta-regressions, intervention duration was not related to changes in incidence risk.

CONCLUSIONS

Trials that have a longer treatment duration and higher doses are needed to clarify whether patients that received adalimumab had an elevated risk of general infection.

The quality of reporting general safety parameters and immune-related adverse events in clinical trials of FDA-approved immune checkpoint inhibitors

BACKGROUND

While immune-checkpoint inhibitors (ICIs) have transformed the field of oncology for advanced-stage cancers, they can lead to serious immune toxicities. Several systematic reviews have evaluated the risk of immune-related adverse events (irAEs); however, most have focused on published articles without evaluating trial registries. The objective of this methodological review was to compare the quality of reporting of safety information and in particular, serious irAEs (irSAEs), in both publications and ClinicalTrials.gov for all current FDA-approved ICIs.

METHODS

PubMed was searched to retrieve all published phase III randomized controlled trials (RCTs) evaluating ICIs. For each eligible trial, we searched for corresponding registration on ClinicalTrials.gov and extracted relevant safety data from both the publication and results posted on registry. We then compared the quality of reporting and the value of safety data between both sources.

RESULTS

Of 42 eligible published trials, 34 had results posted on ClinicalTrials.gov . Considerable variability was noted in the reporting of safety in both sources. SAEs were reported for all trial results in ClinicalTrials.gov compared to 23.5% of publications. An overall incidence for irAEs and irSAEs was reported in 58.8 and 8.8% of publications respectively, compared to 11.8 and 5.9% in registry results. Comparing the value of specific irSAEs was not possible between the two sources in 32/34 trials either due to different reporting formats (61.8%) or data not being reported in one or both sources (32.4%). From the 2 studies with compatible irSAE format, only 1 had matching data in both sources.

CONCLUSIONS

The reporting of irAEs / irSAEs varies considerably in publications and registries, which outlines the importance of standardizing the terminologies and methodologies for reporting safety information relevant to ICIs.

Inferences About Drug Safety in Phase III Trials in Oncology: Examples From Advanced Prostate Cancer

BACKGROUND

Safety is a central consideration when choosing between multiple medications with similar efficacy. We aimed to evaluate whether adverse event (AE) profiles of 3 such drugs in advanced prostate cancer could be distinguished based on published literature.

METHODS

We assessed consistency in AE reporting, AE risk in placebo arms, and methodology used for risk estimates and quantification of statistical uncertainty in randomized placebo-controlled phase III trials of apalutamide, enzalutamide, and darolutamide in advanced prostate cancer.

RESULTS

Seven included clinical trials enrolled a total of 9215 participants (range = 1051-1715 per trial) across 3 prostate cancer disease states. Within disease states, baseline patient characteristics appeared similar between trials. Of 54 distinct AE types in total, only 3 (fatigue, hypertension, and seizure) were reported by all 7 trials. Absolute risks of AEs in the placebo arms differed systematically and more than twofold between trials, which was associated with visit frequency and resulted in different degrees of uncertainty in AE profiles between trials. No trial used inferential methodology to quantify statistical uncertainty in AE risks, but 6 of 7 trials drew overall conclusions. Two trials concluded that there was no elevated AE risk because of the intervention, including the trial of darolutamide, which had the greatest statistical uncertainty.

CONCLUSIONS

Rigorous comparison of drug safety was precluded by heterogeneity in AE reporting, variation in AE risks in the placebo arms, and lack of inferential statistical methodology, underscoring considerable opportunities to improve how AE data are collected, analyzed, and interpreted in oncology trials.

The use of and adherence to CTCAE v3.0 in cancer clinical trial publications

Background

The Common Terminology Criteria for Adverse Events, Version 3.0 (CTCAE v3.0) was released in 2003, and has been widely used as the predominant set of toxicity criteria for cancer clinical trials and scientific meetings. However, the degree to which the elements of CTCAE v3.0 are followed in oncology publications has not been comprehensively evaluated.

Methods

We reviewed phase III randomized clinical trials evaluating systemic cancer therapies, published between Jan 1, 2012 and December 31, 2013, to identify eligible studies that explicitly mentioned using CTCAE v3.0 as the toxicity criteria. A 10-point score based on adherence to CTCAE v3.0 was used to assess the studies. Multivariate linear regression was used to identify features associated with improved adherence.

Results

In total, 104 publications reporting data on 86,957 patients were included in this analysis. The mean total score for adherence to all four elements of CTCAE v3.0 was 4.03 on a 10-point scale (range, 1 to 9), with 16 publications (15%) having total scores ≤2. Highly heterogeneous and unstandardized adverse event terms were frequently used. In addition, Supra-ordinate terms, terms using ‘Other, specify’, and Grades were often used incorrectly. The multivariate regression model revealed that the absence of a placebo (P=0.003) and a higher total number of AE terms in the table (P<0.001) were independent predictors of a lower total score.

Conclusion

Given the importance of understanding the toxicity of new treatments, better adherence to CTCAE v3.0 should be encouraged to ensure the consistency and comparability of toxicity data across different studies.

Time required to review research protocols at 10 Veterans Affairs Institutional Review Boards

BACKGROUND

Despite perceptions that institutional review boards (IRBs) delay research, little is known about how long it takes to secure IRB approval. We retrospectively quantified IRB review times at 10 large Veterans Affairs (VA) IRBs.

METHODS

We collected IRB records pertaining to a stratified random sample of research protocols drawn from 10 of the 26 largest VA IRBs. Two independent analysts abstracted dates from the IRB records, from which we calculated overall and incremental review times. We used multivariable linear regression to assess variation in total and incremental review times by IRB and review level (i.e., exempt, expedited, or full board) and to identify potential targets for efforts to improve the efficiency and uniformity of the IRB review process.

RESULTS

In a sample of 277 protocols, the mean review time was 112 d (95% confidence interval [CI]: 105-120). Compared with full-board reviews at IRB 1, average review times at IRBs 3, 8, 9, and 10 were 27 (95% CI: 6-48), 37 (95% CI: 11-63), 45 (95% CI: 20-69), and 24 (95% CI: 2-45) d shorter, and at IRB 6, times were 56 (95% CI: 28-84) d longer. Across all IRBs, expedited reviews were 44 (95% CI: 30-58) d shorter on average than were full-board reviews, with no significant difference between exempt and full-board reviews. However, after subtracting the time required for Research and Development Committee review, exempt reviews were 21 (95% CI: 1-41) d shorter on average than were full-board reviews.

CONCLUSIONS

IRB review times differ significantly by IRB and review level. Few VA IRBs approach a consensus panel goal of 60 d for IRB review. The unexpectedly longer review times for exempt protocols in the VA can be attributed to time required for Research and Development Committee review. Prospective, routine collection of key time points in the IRB review process could inform IRB-specific initiatives for reducing VA IRB review times.

Assessment of Adverse Events in Protocols, Clinical Study Reports, and Published Papers of Trials of Orlistat: A Document Analysis

BACKGROUND

Little is known about how adverse events are summarised and reported in trials, as detailed information is usually considered confidential. We have acquired clinical study reports (CSRs) from the European Medicines Agency through the Freedom of Information Act. The CSRs describe the results of studies conducted as part of the application for marketing authorisation for the slimming pill orlistat. The purpose of this study was to study how adverse events were summarised and reported in study protocols, CSRs, and published papers of orlistat trials.

METHODS AND FINDINGS

We received the CSRs from seven randomised placebo controlled orlistat trials (4,225 participants) submitted by Roche. The CSRs consisted of 8,716 pages and included protocols. Two researchers independently extracted data on adverse events from protocols and CSRs. Corresponding published papers were identified on PubMed and adverse event data were extracted from this source as well. All three sources were compared. Individual adverse events from one trial were summed and compared to the totals in the summary report. None of the protocols or CSRs contained instructions for investigators on how to question participants about adverse events. In CSRs, gastrointestinal adverse events were only coded if the participant reported that they were "bothersome," a condition that was not specified in the protocol for two of the trials. Serious adverse events were assessed for relationship to the drug by the sponsor, and all adverse events were coded by the sponsor using a glossary that could be updated by the sponsor. The criteria for withdrawal due to adverse events were in one case related to efficacy (high fasting glucose led to withdrawal), which meant that one trial had more withdrawals due to adverse events in the placebo group. Finally, only between 3% and 33% of the total number of investigator-reported adverse events from the trials were reported in the publications because of post hoc filters, though six of seven papers stated that "all adverse events were recorded." For one trial, we identified an additional 1,318 adverse events that were not listed or mentioned in the CSR itself but could be identified through manually counting individual adverse events reported in an appendix. We discovered that the majority of patients had multiple episodes of the same adverse event that were only counted once, though this was not described in the CSRs. We also discovered that participants treated with orlistat experienced twice as many days with adverse events as participants treated with placebo (22.7 d versus 14.9 d, p-value < 0.0001, Student's t test). Furthermore, compared with the placebo group, adverse events in the orlistat group were more severe. None of this was stated in the CSR or in the published paper. Our analysis was restricted to one drug tested in the mid-1990s; our results might therefore not be applicable for newer drugs.

CONCLUSIONS

In the orlistat trials, we identified important disparities in the reporting of adverse events between protocols, clinical study reports, and published papers. Reports of these trials seemed to have systematically understated adverse events. Based on these findings, systematic reviews of drugs might be improved by including protocols and CSRs in addition to published articles.

Prioritizing Initiatives for Institutional Review Board (IRB) Quality Improvement

Background

Institutional Review Boards (IRBs) have been criticized for inconsistency, delay, and bias, suggesting an opportunity for quality improvement. To aid such quality improvement, this study aimed at determining IRB members' and investigators' priorities regarding IRB review at 10 Veterans Affairs (VA) IRBs.

Methods

680 anonymous internet surveys were sent to 252 IRB members and staff, and 428 principal investigators and project coordinators at 9 VA Medical Centers and the VA Central IRB. Surveys included 27 statements adapted from Koocher and Kieth-Spiegel's IRB-RAT describing IRB activities or functions (e.g., "An IRB that is open to reversing its earlier decisions"). Respondents indicated how each statement described both their "ideal" and "actual" IRBs. The difference between the ratings of the actual and ideal IRBs was calculated for each item along with estimated 95% confidence intervals. Ratings outside those intervals indicated activities or functions with relatively good or poor performance compared to the ideal IRB.

Results

390 (57.4%) responses from 165 IRB members and staff (65.5%) and 225 investigators and project coordinators (52.6%) demonstrated that these IRBs were closest to the ideal when protecting human subjects, treating investigators with respect, and taking appropriate action for alleged scientific misconduct. The IRBs were furthest from the ideal regarding duplicative forms, timeliness of review, and provision of complete rationales for decisions. Although IRB members reported near-ideal willingness to reverse earlier decisions, investigators rated this capacity far from ideal. Investigators rated IRB members' knowledge about procedures and policy as nearly ideal, but IRB members themselves rated this aspect far from ideal. Noteworthy site-level differences in the ratings of IRB functions and activities were also identified.

Conclusions

Although these VA IRBs perform well in some areas, these data support the need for ongoing quality improvement. The described method of administering and analyzing the IRB-RAT may help identify and monitor site- and activity- specific initiatives for quality improvement.

Cost-benefit assessment of using electronic health records data for clinical research versus current practices: Contribution of the Electronic Health Records for Clinical Research (EHR4CR) European Project

INTRODUCTION

The widespread adoption of electronic health records (EHR) provides a new opportunity to improve the efficiency of clinical research. The European EHR4CR (Electronic Health Records for Clinical Research) 4-year project has developed an innovative technological platform to enable the re-use of EHR data for clinical research. The objective of this cost-benefit assessment (CBA) is to assess the value of EHR4CR solutions compared to current practices, from the perspective of sponsors of clinical trials.

MATERIALS AND METHODS

A CBA model was developed using an advanced modeling approach. The costs of performing three clinical research scenarios (S) applied to a hypothetical Phase II or III oncology clinical trial workflow (reference case) were estimated under current and EHR4CR conditions, namely protocol feasibility assessment (S1), patient identification for recruitment (S2), and clinical study execution (S3). The potential benefits were calculated considering that the estimated reduction in actual person-time and costs for performing EHR4CR S1, S2, and S3 would accelerate time to market (TTM). Probabilistic sensitivity analyses using Monte Carlo simulations were conducted to manage uncertainty.

RESULTS

Should the estimated efficiency gains achieved with the EHR4CR platform translate into faster TTM, the expected benefits for the global pharmaceutical oncology sector were estimated at €161.5m (S1), €45.7m (S2), €204.5m (S1+S2), €1906m (S3), and up to €2121.8m (S1+S2+S3) when the scenarios were used sequentially.

CONCLUSIONS

The results suggest that optimizing clinical trial design and execution with the EHR4CR platform would generate substantial added value for pharmaceutical industry, as main sponsors of clinical trials in Europe, and beyond.

Latent variable modeling and its implications for institutional review board review: variables that delay the reviewing process

Background

To investigate the factors related to approval after review by an Institutional Review Board (IRB), the structure equation model was used to analyze the latent variables ‘investigators’, ‘vulnerability’ and ‘review process’ for 221 proposals submitted to our IRB.

Methods

The vulnerability factor included vulnerable cases, and studies that involved drug tests and genetic analyses. The principal investigator (PI) factor included the license level of the PI and whether they belonged to our institution. The review factor included administration time, total review time, and revision frequency. The revision frequency and total review time influenced the efficiency of review.

Results

The latent variable of reviewing was the most important factor mediating the PIs and vulnerability to IRB review approval. The local PIs moderated with genetic study and revision frequency had an impact on the review process and mediated non-approval.

Conclusions

Better guidance of the investigators and reviewers might improve the efficiency with which IRBs function.

Time required for institutional review board review at one Veterans Affairs medical center

IMPORTANCE

Despite growing concern that institutional review boards (IRBs) impose burdensome delays on research, little is known about the time required for IRB review across different types of research.

OBJECTIVE

To measure the overall and incremental process times for IRB review as a process of quality improvement.

DESIGN, SETTING, AND PARTICIPANTS

After developing a detailed process flowchart of the IRB review process, 2 analysts abstracted temporal data from the records pertaining to all 103 protocols newly submitted to the IRB at a large urban Veterans Affairs medical center from June 1, 2009, through May 31, 2011. Disagreements were reviewed with the principal investigator to reach consensus. We then compared the review times across review types using analysis of variance and post hoc Scheffé tests after achieving normally distributed data through logarithmic transformation.

MAIN OUTCOMES AND MEASURES

Calendar days from initial submission to final approval of research protocols.

RESULTS

Initial IRB review took 2 to 4 months, with expedited and exempt reviews requiring less time (median [range], 85 [23-631] and 82 [16-437] days, respectively) than full board reviews (median [range], 131 [64-296] days; P = .008). The median time required for credentialing of investigators was 1 day (range, 0-74 days), and review by the research and development committee took a median of 15 days (range, 0-184 days). There were no significant differences in credentialing or research and development times across review types (exempt, expedited, or full board). Of the extreme delays in IRB review, 80.0% were due to investigators' slow responses to requested changes. There were no systematic delays attributable to the information security officer, privacy officer, or IRB chair.

CONCLUSIONS AND RELEVANCE

Measuring and analyzing review times is a critical first step in establishing a culture and process of continuous quality improvement among IRBs that govern research programs. The review times observed at this IRB are substantially longer than the 60-day target recommended by expert panels. The method described here could be applied to other IRBs to begin identifying and improving inefficiencies.

Adverse reactions to oncologic drugs: spontaneous reporting and signal detection

Oncology is one of the areas of medicine with the most active research being conducted on new drugs. New pharmacological entities frequently enter the clinical arena, and therefore, the safety profile of anticancer products deserves continuous monitoring. However, only very severe and (unusual) suspected adverse drug reactions (ADRs) are usually reported, since cancer patients develop ADRs very frequently and some practical selectivity must be used. Notably, a recent study was able to identify 76 serious ADRs reported in updated drug labels of oncologic drugs and 50% of them (n = 38) were potentially fatal. Of these, 49 and 58%, respectively, were not described in initial drug labels. The aims of this article are to provide an overview about spontaneous reporting of ADRs of oncologic drugs and to discuss the available methods to analyze the safety of anticancer drugs using databases of spontaneous ADR reporting.

Risk of incremental toxicities and associated costs of new anticancer drugs: a meta-analysis

PURPOSE

There are increasing reports of rare but serious toxicities caused by new anticancer drugs, and there are costs associated with their management.

METHODS

We identified anticancer drugs approved by the U.S. Food and Drug Administration from 2000 to 2011 and pivotal trials supporting their registration. Twelve frequent grade 3 to 4 adverse event (AEs) were weighted and pooled in a meta-analysis. Estimates of incremental drug prices and incremental costs for management of AEs were calculated according to type of new agent based on target specificity.

RESULTS

We identified 41 studies comprising 27,539 patients and evaluating 19 experimental drugs. Agents directed against a specific molecular target on cancer cells had a lower incidence of grade 3 to 4 toxicities compared with controls (median risk ratio [RR], 0.67; P = .22), whereas less-specific targeted agents, including angiogenesis inhibitors (median RR, 3.39; P < .001) and chemotherapeutic agents (median RR, 1.73; P < .001), were more toxic. Risk was increased regardless of whether the control arm contained active treatment (RR, 2.11; P < .001) or not (RR, 3.02; P < .001). Median incremental drug price for experimental agents was $6,000 per patient per month. Median cost of managing toxicity was low compared with drug costs but higher than controls for treatment with less-specific targeted agents and chemotherapies.

CONCLUSION

Newly approved anticancer drugs are associated with increased toxicity, except for agents with a specific molecular target on cancer cells. Management of toxicity leads to a small increase in overall cost of treatment. Frequency of toxicity and associated costs are likely higher in less-selected patients treated in general oncologic practice. Development of biomarker-driven agents should be encouraged.

Adverse Event Reporting in Cancer Clinical Trial Publications

Purpose

Reporting adverse events is a critical element of a clinical trial publication. In 2003, the Consolidated Standards of Reporting Trials (CONSORT) group generated recommendations regarding the appropriate reporting of adverse events. The degree to which these recommendations are followed in oncology publications has not been comprehensively evaluated.

Methods

A review of citations from PubMed, Medline, and Embase published between Jan 1, 2009 and December 31, 2011, identified eligible randomized, controlled phase III trials in metastatic solid malignancies. Publications were assessed for 14 adverse event–reporting elements derived from the CONSORT harms extension statement; a completeness score (range, 0 to 14) was calculated by adding the number of elements reported. Linear regression analysis identified which publication characteristics associated with reporting completeness.

Results

A total of 175 publications, with data for 96,125 patients, were included in the analysis. The median completeness score was eight (range, three to 12). Most publications (96%) reported only adverse events occurring above a threshold rate or severity, 37% did not specify the criteria used to select which adverse events were reported, and 88% grouped together adverse events of varying severity. Regression analysis revealed that trials without a stated funding source and with an earlier year of publication had significantly lower completeness scores.

Conclusion

Reporting of adverse events in oncology publications of randomized trials is suboptimal and characterized by substantial selectivity and heterogeneity. The development of oncology-specific standards for adverse event reporting should be established to ensure consistency and provide critical information required for medical decision-making.

Acute kidney injury and bisphosphonate use in cancer: a report from the research on adverse drug events and reports (RADAR) project

PURPOSE

To determine whether acute kidney injury (AKI) is identified within the US Food and Drug Administration's Adverse Events and Reporting System (FDA AERS) as an adverse event resulting from bisphosphonate (BP) use in cancer therapy.

METHODS

A search of the FDA AERS records from January 1998 through June 2009 was performed; search terms were "renal problems" and all drug names for BPs. The search resulted in 2,091 reports. We analyzed for signals of disproportional association by calculating the proportional reporting ratio for zoledronic acid (ZOL) and pamidronate. Literature review of BP-associated renal injury within the cancer setting was conducted.

RESULTS

Four hundred eighty cases of BP-associated acute kidney injury (AKI) were identified in patients with cancer. Two hundred ninety-eight patients (56%) were female; mean age was 66 ± 10 years. Multiple myeloma (n = 220, 46%), breast cancer (n = 98, 20%), and prostate cancer (n = 24, 5%) were identified. Agents included ZOL (n = 411, 87.5%), pamidronate (n = 8, 17%), and alendronate (n = 36, 2%). Outcomes included hospitalization (n = 304, 63.3%) and death (n = 68, 14%). The proportional reporting ratio for ZOL was 1.22 (95% CI, 1.13 to 1.32) and for pamidronate was 1.55 (95% CI, 1.25 to 1.65), reflecting a nonsignificant safety signal for both drugs.

CONCLUSION

AKI was identified in BP cancer clinical trials, although a safety signal for BPs and AKI within the FDA AERS was not detected. Our findings may be attributed, in part, to clinicians who believe that AKI occurs infrequently; ascribe the AKI to underlying premorbid disease, therapy, or cancer progression; or consider that AKI is a known adverse drug reaction of BPs and thus under-report AKI to the AERS.