Evaluation of the Inclusion of Studies Identified by the FDA as Having Falsified Data in the Results of Meta-analyses: The Example of the Apixaban Trials

Reducing the residue of retractions in evidence synthesis: ways to minimise inappropriate citation and use of retracted data

The incorporation of publications that have been retracted is a risk in reliable evidence synthesis. Retraction is an important mechanism for correcting the literature and protecting its integrity. Within the medical literature, the continued citation of retracted publications occurs for a variety of reasons. Recent evidence suggests that systematic reviews and meta-analyses often unwittingly cite retracted publications which, at least in some cases, may significantly impact quantitative effect estimates in meta-analyses. There is strong evidence that authors of systematic reviews and meta-analyses may be unaware of the retracted status of publications and treat them as if they are not retracted. These problems are difficult to address for several reasons: identifying retracted publications is important but logistically challenging; publications may be retracted while a review is in preparation or in press and problems with a publication may also be discovered after the evidence synthesis is published. We propose a set of concrete actions that stakeholders (eg, scientists, peer-reviewers, journal editors) might take in the near-term, and that research funders, citation management systems, and databases and search engines might take in the longer term to limit the impact of retracted primary studies on evidence syntheses.

The relationship between methodological quality and the use of retracted publications in evidence syntheses

BACKGROUND

Evidence syntheses cite retracted publications. However, citation is not necessarily endorsement, as authors may be criticizing or refuting its findings. We investigated the sentiment of these citations-whether they were critical or supportive-and associations with the methodological quality of the evidence synthesis, reason for the retraction, and time between publication and retraction.

METHODS

Using a sample of 286 evidence syntheses containing 324 citations to retracted publications in the field of pharmacy, we used AMSTAR-2 to assess methodological quality. We used scite.ai and a human screener to determine citation sentiment. We conducted a Pearson's chi-square test to assess associations between citation sentiment, methodological quality, and reason for retraction, and one-way ANOVAs to investigate association between time, methodological quality, and citation sentiment.

RESULTS

Almost 70% of the evidence syntheses in our sample were of critically low quality. We found that these critically low-quality evidence syntheses were more associated with positive statements while high-quality evidence syntheses were more associated with negative citation of retracted publications. In our sample of 324 citations, 20.4% of citations to retracted publications noted that the publication had been retracted.

CONCLUSION

The association between high-quality evidence syntheses and recognition of a publication's retracted status may indicate that best practices are sufficient. However, the volume of critically low-quality evidence syntheses ultimately perpetuates the citation of retracted publications with no indication of their retracted status. Strengthening journal requirements around the quality of evidence syntheses may lessen the inappropriate citation of retracted publications.

Assessing the robustness of results from clinical trials and meta-analyses with the fragility index

The fragility index has been increasingly used to assess the robustness of the results of clinical trials since 2014. It aims at finding the smallest number of event changes that could alter originally statistically significant results. Despite its popularity, some researchers have expressed several concerns about the validity and usefulness of the fragility index. It offers a comprehensive review of the fragility index's rationale, calculation, software, and interpretation, with emphasis on application to studies in obstetrics and gynecology. This article presents the fragility index in the settings of individual clinical trials, standard pairwise meta-analyses, and network meta-analyses. Moreover, this article provides worked examples to demonstrate how the fragility index can be appropriately calculated and interpreted. In addition, the limitations of the traditional fragility index and some solutions proposed in the literature to address these limitations were reviewed. In summary, the fragility index is recommended to be used as a supplemental measure in the reporting of clinical trials and a tool to communicate the robustness of trial results to clinicians. Other considerations that can aid in the fragility index's interpretation include the loss to follow-up and the likelihood of data modifications that achieve the loss of statistical significance.

Methods for living guidelines: early guidance based on practical experience. Article 5: decisions on methods for evidence synthesis and recommendation development for living guidelines

OBJECTIVES

Producing living guidelines requires making important decisions about methods for evidence identification, appraisal, and integration to allow the living mode to function. Clarifying what these decisions are and the trade-offs between options is necessary. This article provides living guideline developers with a framework to enable them to choose the most suitable model for their living guideline topic, question, or context.

STUDY DESIGN AND SETTING

We developed this guidance through an iterative process informed by interviews, feedback, and a consensus process with an international group of living guideline developers.

RESULTS

Several key decisions need to be made both before commencing and throughout the continual process of living guideline development and maintenance. These include deciding what approach is taken to the systematic review process; decisions about methods to be applied for the evidence appraisal process, including the use of unpublished data; and selection of "triggers" to incorporate new studies into living guideline recommendations. In each case, there are multiple options and trade-offs.

CONCLUSION

We identify trade-offs and important decisions to be considered throughout the living guideline development process. The most appropriate, and most sustainable, mode of development and updating will be dependent on the choices made in each of these areas.

Replacement of traditional prothrombin time monitoring with the new Fiix prothrombin time increases the efficacy of warfarin without increasing bleeding. A review article

The antithrombotic effect of vitamin K antagonists (VKA) depends on controlled lowering of the activity of factors (F) II and X whereas reductions in FVII and FIX play little role. PT-INR based monitoring, however, is highly influenced by FVII, which has the shortest half-life of vitamin K-dependent coagulation factors. Hence, variability in the anticoagulant effect of VKA may be partly secondary to an inherent flaw of the traditional monitoring test itself. The Fiix prothrombin time (Fiix-PT) is a novel test that is only sensitive to reductions in FII and FX and is intended to stabilize the VKA effect. Two clinical studies have now demonstrated that when warfarin is monitored with the Fiix-PT based normalized ratio (Fiix-NR) instead of PT-INR, anticoagulation is stabilized and less testing and fewer dose adjustments are needed. Furthermore, the relative risk of thromboembolism was reduced by 50-56% in these studies without an increase in major bleeding.

What difference might retractions make? An estimate of the potential epistemic cost of retractions on meta-analyses

The extent to which a retraction might require revising previous scientific estimates and beliefs - which we define as the epistemic cost - is unknown. We collected a sample of 229 meta-analyses published between 2013 and 2016 that had cited a retracted study, assessed whether this study was included in the meta-analytic estimate and, if so, re-calculated the summary effect size without it. The majority (68% of N = 229) of retractions had occurred at least one year prior to the publication of the citing meta-analysis. In 53% of these avoidable citations, the retracted study was cited as a candidate for inclusion, and only in 34% of these meta-analyses (13% of total) the study was explicitly excluded because it had been retracted. Meta-analyses that included retracted studies were published in journals with significantly lower impact factor. Summary estimates without the retracted study were lower than the original if the retraction was due to issues with data or results and higher otherwise, but the effect was small. We conclude that meta-analyses have a problematically high probability of citing retracted articles and of including them in their pooled summaries, but the overall epistemic cost is contained.

A Systematic Review of the Efficacy and Safety of Direct Oral Anticoagulants in Atrial Fibrillation Patients with Diabetes Using a Risk Index

Diabetes mellitus (DM) represents an independent risk factor for chronic AF and is associated with unfavorable outcomes. We aimed to evaluate the efficacy and safety of direct oral anticoagulants (DOACs) in patients with atrial fibrillation (AF), with and without diabetes mellitus (DM), using a new risk index (RI) defined as: RI =Rate of EventsRate of Patients at Risk. In particular, an RI lower than 1 suggests a favorable treatment effect. We searched MEDLINE, MEDLINE In-Process, EMBASE, PubMed, and the Cochrane Central Register of Controlled Trials. The risk index (RI) was calculated in terms of efficacy (rate of stroke/systemic embolism (stroke SEE)/rate of patients with and without DM; rate of cardiovascular death/rate of patients with and without DM) and safety (rate of major bleeding/rate of patients with and without DM) outcomes. AF patients with DM (n = 22,057) and 49,596 without DM were considered from pivotal trials. DM doubles the risk index for stroke/SEE, major bleeding (MB), and cardiovascular (CV) death. The RI for stroke/SEE, MB, and CV death was comparable in patients treated with warfarin or DOACs. The lowest RI was in DM patients treated with Rivaroxaban (stroke/SEE, RI = 0.08; CV death, RI = 0.13). The RIs for bleeding were higher in DM patients treated with Dabigatran (RI110 = 0.32; RI150 = 0.40). Our study is the first to use RI to homogenize the efficacy and safety data reported in the DOACs pivotal studies against warfarin in patients with and without DM. Anticoagulation therapy is effective and safe in DM patients. DOACs appear to have a better efficacy and safety profile than warfarin. The use of DOACs is a reasonable alternative to vitamin-K antagonists in AF patients with DM. The RI can be a reasonable tool to help clinicians choose between DOACs or warfarin in the peculiar set of AF patients with DM.

Ignoring instead of chasing after coagulation factor VII during warfarin management: an interrupted time series study

During warfarin management, variability in prothrombin time-based international normalized ratio (PT-INR) is caused, in part, by clinically inconsequential fluctuations in factor VII (FVII). The new factor II and X (Fiix)-prothrombin time (Fiix-PT) and Fiix-normalized ratio (Fiix-NR), unlike PT-INR, are only affected by reduced FII and FX. We assessed the incidence of thromboembolism (TE) and major bleeding (MB) in all 2667 patients on maintenance-phase warfarin managed at our anticoagulation management service during 30 months; 12 months prior to and 18 months after replacing PT-INR monitoring with Fiix-NR monitoring. Months 13 to 18 were predefined as transitional months. Using 2-segmented regression, a breakpoint in the monthly incidence of TE became evident 6 months after test replacement, that was followed by a 56% reduction in incidence (from 2.82% to 1.23% per patient-year; P = .019). Three-segmented regression did not find any significant trend in TE incidence (slope, +0.03) prior to test replacement; however, during months 13 to 18 and 19 to 30, the incidence of TE decreased gradually (slope, -0.12; R2 = 0.20; P = .007). The incidence of MB (2.79% per patient-year) did not differ. Incidence comparison during the 12-month Fiix and PT periods confirmed a statistically significant reduction (55-62%) in TE. Fiix monitoring reduced testing, dose adjustments, and normalized ratio variability and prolonged testing intervals and time in range. We conclude that ignoring FVII during Fiix-NR monitoring in real-world practice stabilizes the anticoagulant effect of warfarin and associates with a major reduction in TEs without increasing bleeding.

For how long and with what relevance do genetics articles retracted due to research misconduct remain active in the scientific literature

We aimed to quantify the number of pre- and post-retraction citations obtained by genetics articles retracted due to research misconduct. All retraction notices available in the Retraction Watch database for genetics articles published in 1970-2016 were assessed. The reasons for retraction were fabrication/falsification and plagiarism. The endpoints were the number of citations of retracted articles and when and how journals reported on retractions and whether this was published on PubMed.Four hundred and sixty retracted genetics articles were cited 34,487 times; 7,945 (23%) were post-retraction citations. Median time to retraction and time to last citation were 3.2 and 3 years, respectively. Most (96%) had a PubMed retraction notice, One percent of these were totally removed from journal websites altogether, and 4% had no information available on either the online or PDF versions. Ninety percent of citations were from articles retracted due to falsification/fabrication. The percentage of post-retraction citations was significantly higher in the case of plagiarism (42%) than in the case of fabrication/falsification (21.5%) (p<0.001). Median time to retraction was shorter (1.3 years) in the case of plagiarism than for fabrication/falsification (4.8 years, p<0.001). The retraction was more frequently reported in the PDFs (70%) for the fabrication/falsification cases than for the plagiarism cases (43%, p<0.001). The highest rate of retracted papers due to falsification/fabrication was among authors in the USA, and the highest rate for plagiarism was in China.Although most retractions were appropriately handled by journals, the gravest issue was that median time to retraction for articles retracted for falsification/fabrication was nearly 5 years, earning close to 6800 post-retraction citations. Journals should implement processes to speed-up the retraction process that will help to minimize post-retraction citations.

Perspective on Systematic Medical Literature Reviews and Meta-Analyses

PURPOSE

This study sought to identify factors contributing to the inadequacies of systematic reviews and meta-analyses (SRMAs) published in the ophthalmology literature.

DESIGN

Perspective.

METHODS

Review and synthesis of selective literature, with interpretation and perspective.

RESULTS

Although recommendations for the design, conduct, assessment of quality, and risk of bias of systematic reviews have been widely available, some recent publications illustrate a serious potential failing in this domain: inclusion of refuted science, lack of citation of post-publication correspondence and failure to use ≥1 alternative search strategy.

CONCLUSIONS

Examples of inadequacies of peer review in medical literature and perpetuation of erroneous science by unfiltered inclusion in subsequent systematic reviews have been identified, and the problem can be traced to authors, peer reviewers, and editors of journals. This perspective identifies and analyzes several possible causes of the problem and recommends some specific corrective actions to improve the quality and accuracy of such reviews.

Pharmaceutical industry funding of events for healthcare professionals on non-vitamin K oral anticoagulants in Australia: an observational study

OBJECTIVES

To describe the nature, frequency and content of non-vitamin K oral anticoagulant (NOAC)-related events for healthcare professionals sponsored by the manufacturers of the NOACs in Australia. A secondary objective is to compare these data to the rate of dispensing of the NOACs in Australia.

DESIGN AND SETTING

This cross-sectional study examined consolidated data from publicly available Australian pharmaceutical industry transparency reports from October 2011 to September 2015 on NOAC-related educational events. Data from April 2011 to June 2016 on NOAC dispensing, subsidised under Australia's Pharmaceutical Benefits Scheme (PBS), were obtained from the Department of Health and the Department of Human Services.

MAIN OUTCOME MEASURES

Characteristics of NOAC-related educational events including costs (in Australian dollars, $A), numbers of events, information on healthcare professional attendees and content of events; and NOAC dispensing rates.

RESULTS

During the study period, there were 2797 NOAC-related events, costing manufacturers a total of $A10 578 745. Total expenditure for meals and beverages at all events was $A4 238 962. Events were predominantly attended by general practitioners (42%, 1174/2797), cardiologists (35%, 977/2797) and haematologists (23%, 635/2797). About 48% (1347/2797) of events were held in non-clinical settings, mainly restaurants, bars and cafes. Around 55% (1551/2797) of events consisted of either conferences, meetings or seminars. The analysis of the content presented at two events detected promotion of NOACs for unapproved indications, an emphasis on a favourable benefit/harm profile, and that all speakers had close ties with the manufacturers of the NOACs. Following PBS listings relevant to each NOAC, the numbers of events related to that NOAC and the prescribing of that NOAC increased.

CONCLUSIONS

Our findings suggest that the substantial investment in NOAC-related events made by four pharmaceutical companies had a promotional purpose. Healthcare professionals should seek independent information on newly subsidised medicines from, for example, government agencies or drug bulletins.