Multiple overlapping systematic reviews facilitate the origin of disputes: the case of thrombolytic therapy for pulmonary embolism

Definition, harms, and prevention of redundant systematic reviews

BACKGROUND

Along with other types of research, it has been stated that the extent of redundancy in systematic reviews has reached epidemic proportions. However, it was also emphasized that not all duplication is bad, that replication in research is essential, and that it can help discover unfortunate behaviors of scientists. Thus, the question is how to define a redundant systematic review, the harmful consequences of such reviews, and what we could do to prevent the unnecessary amount of this redundancy.

MAIN BODY

There is no consensus definition of a redundant systematic review. Also, it needs to be defined what amount of overlap between systematic reviews is acceptable and not considered a redundancy. One needs to be aware that it is possible that the authors did not intend to create a redundant systematic review. A new review on an existing topic, which is not an update, is likely justified only when it can be shown that the previous review was inadequate, for example, due to suboptimal methodology. Redundant meta-analyses could have scientific, ethical, and economic questions for researchers and publishers, and thus, they should be avoided, if possible. Potential solutions for preventing redundant reviews include the following: (1) mandatory prospective registration of systematic reviews; (2) editors and peer reviewers rejecting duplicate/redundant and inadequate reviews; (3) modifying the reporting checklists for systematic reviews; (4) developing methods for evidence-based research (EBR) monitoring; (5) defining systematic reviews; (6) defining the conclusiveness of systematic reviews; (7) exploring interventions for the adoption of methodological advances; (8) killing off zombie reviews (i.e., abandoned registered reviews); (9) better prevention of duplicate reviews at the point of registration; (10) developing living systematic reviews; and (11) education of researchers.

CONCLUSIONS

Disproportionate redundancy of the same or very similar systematic reviews can lead to scientific, ethical, economic, and societal harms. While it is not realistic to expect that the creation of redundant systematic reviews can be completely prevented, some preventive measures could be tested and implemented to try to reduce the problem. Further methodological research and development in this field will be welcome.

Massive pulmonary embolism and intra-cardiac thrombus requiring systemic thrombolysis 9-hours post emergency laparotomy

The link between abdominal surgery and venous thromboembolism (VTE) has been well established with recent evidence exploring the optimal VTE risk reducing strategy. However, despite these strategies pulmonary embolisms (PEs) do occur, which in the immediate post-operative setting creates a dilemma; to treat the VTE with anticoagulation but balance against the risk of hemorrhage. Treatment guidelines often do not include post-operative patients leaving the decision up to the treating physician to weigh the relative risks on an individual basis. We present a 59-year-old lady who developed a life-threatening submassive PE within 9 h of an emergency laparotomy for a perforated rectal cancer. She was treated with systemic thrombolysis after alternative interventions had been excluded. She responded well to therapy with no major bleeding. She was successfully discharged home after a short period of inpatient rehabilitation.

Ultra-Low-Dose Systemic Tissue Plasminogen Activator in High-Risk Submassive Pulmonary Embolism

Risk stratification of pulmonary embolism (PE) is vital for clinical management. While low-risk and high-risk PE management are clearly defined in many societal guidelines, the management of moderate-risk, also called submassive, PE remains unsettled. There is a subgroup of patients with submassive PE that progress to the severe category despite receiving systemic anticoagulation. The role of thrombolysis in the management of submassive PE remains to be established. We share our experience with ultra-low-dose (25-mg) systemic tissue plasminogen activator in a series of 4 patients with high-risk submassive PE.

Identifying and addressing conflicting results across multiple discordant systematic reviews on the same question: protocol for a replication study of the Jadad algorithm

INTRODUCTION

An increasing growth of systematic reviews (SRs) presents notable challenges for decision-makers seeking to answer clinical questions. In 1997, an algorithm was created by Jadad to assess discordance in results across SRs on the same question. Our study aims to (1) replicate assessments done in a sample of studies using the Jadad algorithm to determine if the same SR would have been chosen, (2) evaluate the Jadad algorithm in terms of utility, efficiency and comprehensiveness, and (3) describe how authors address discordance in results across multiple SRs.

METHODS AND ANALYSIS

We will use a database of 1218 overviews (2000-2020) created from a bibliometric study as the basis of our search for studies assessing discordance (called discordant reviews). This bibliometric study searched MEDLINE (Ovid), Epistemonikos and Cochrane Database of Systematic Reviews for overviews. We will include any study using Jadad (1997) or another method to assess discordance. The first 30 studies screened at the full-text stage by two independent reviewers will be included. We will replicate the authors' Jadad assessments. We will compare our outcomes qualitatively and evaluate the differences between our Jadad assessment of discordance and the authors' assessment.

ETHICS AND DISSEMINATION

No ethics approval was required as no human subjects were involved. In addition to publishing in an open-access journal, we will disseminate evidence summaries through formal and informal conferences, academic websites, and across social media platforms. This is the first study to comprehensively evaluate and replicate Jadad algorithm assessments of discordance across multiple SRs.

Mitigating Disputes Originated by Multiple Discordant Systematic Reviews and Meta-Analyses: A Survey of Methodologists and Clinicians

Background

Overlapping systematic reviews (SRs) are increasingly frequent in the medical literature. They can easily generate discordant evidence, as estimates of effect sizes and their interpretation might differ from one source to another.

Objective

To analyze how methodologists and clinicians make a decision when faced with discordant evidence formalized in structured tables.

Methods

We conducted a 16-item survey exploring how methodologists and clinicians would react when presented with multiple Summary of Findings (SoF) tables (generated using the GRADE tool) derived from 4 overlapping and discordant SRs and meta-analyses on thrombolytic therapy for intermediate-risk pulmonary embolism. SoF tables reported 4 different magnitudes of effects and overall certainty. Participants were asked to provide their recommendations regarding the intervention and the reasons behind their conclusion.

Results

Of the 80 invitees, 41 (51%) participated. The majority described themselves as “somewhat familiar” or experts with SoF tables. The majority recommended the therapy (pharmacological systemic thrombolysis), grading the recommendation as weak positive. Certainty of evidence and benefit-risk balance were the two criteria that prevailed in generating the recommendation. When faced with overlapping meta-analyses, the preferred approach was to use only high-quality SRs and exclude redundant SRs. Several participants suggested integrating the SoF tables with additional information, such as a more comprehensive evaluation of the risk of bias of systematic reviews (71%), heterogeneity/inconsistency (68%) and studies included within each SR (62%).

Conclusion

When faced with multiple controversial SR results, the type and completeness of reported information in SoF tables affect experts' ability to make recommendations. Developers of the SoF table should consider collating key information from overlapping and potentially discordant reviews.

Submassive Pulmonary Embolism: Current Perspectives and Future Directions

Submassive pulmonary embolism (PE) lies on a spectrum of disease severity between standard and high-risk disease. By definition, patients with submassive PE have a worse outcome than the majority of those with standard-risk PE, who are hemodynamically stable and lack imaging or laboratory features of cardiac dysfunction. Systemic thrombolytic therapy has been proven to reduce mortality in patients with high-risk disease; however, its use in submassive PE has not demonstrated a clear benefit, with haemodynamic improvements being offset by excess bleeding. Furthermore, meta-analyses have been confusing, with conflicting results on overall survival and net gain. As such, significant interest remains in optimising thrombolysis, with recent efforts in catheter-based delivery as well as upcoming studies on reduced systemic dosing. Recently, long-term cardiorespiratory limitations following submassive PE have been described, termed post-PE syndrome. Studies on the ability of thrombolytic therapy to prevent this condition also present conflicting evidence. In this review, we aim to clarify the current evidence with respect to submassive PE management, and also to highlight shortcomings in current definitions and prognostic factors. Additionally, we discuss novel therapies currently in preclinical and early clinical trials that may improve outcomes in patients with submassive PE.

Mechanisms of thrombosis and research progress on targeted antithrombotic drugs

Globally, the incidence of thromboembolic diseases has increased in recent years, accompanied by an increase in patient mortality. Currently, several targeting delivery strategies have been developed to treat thromboembolic diseases. In this review, we discuss the mechanisms of thrombolysis and current anticoagulant drugs, particularly those with targeting capability, highlighting advances in the accurate treatment of thrombolysis with fewer adverse effects. Such approaches include magnetic drug-loading systems combined with molecular imaging to recanalize blood vessels and systems based on chimeric Arg-Gly-Asp (RGD) sequences that can target platelet glycoprotein receptor. With such progress in targeted antithrombotic drugs, targeted thrombolysis treatment shows significant potential benefit for patients.

The net benefit of thrombolysis in the management of intermediate risk pulmonary embolism: Systematic review and meta‐analysis

Background

Benefit of thrombolytic therapy in patients with massive pulmonary embolism (PE) is evident. However, evidence supporting benefit in clinical outcomes of this approach in intermediate risk PE is lacking.

Objective

To determine the impact of thrombolysis on overall survival in intermediate risk PE patients.

Methods

We searched in MEDLINE (OVID), EMBASE, LILACS, and the Cochrane Central Register of Controlled Trials (CENTRAL) from present day. We also searched in other databases and unpublished literature. We included clinical trials without language restrictions. The risk of bias was evaluated with the Cochrane Collaboration's tool. The primary outcome was overall survival. Secondary outcomes were adverse events, including major bleeding, and all‐cause mortality. The measure of the effect was the risk ratio with a 95% confidence interval (CI).

Results

We included 11 studies in the qualitative and quantitative analysis, with a total of 1855 patients. Risk of bias was variable among the study items. There were no results reported about overall survival in any of the studies. The risk ratio (RR) for all‐cause mortality was 0.68 95% CI (0.40 to 1.16). The RR of overall bleeding, major bleeding and stroke were 2.72 95% CI (1.58 to 4.69), 2.17 95% CI (1.03 to 4.55), and 2.22 95% CI (0.17 to 28.73), respectively. Additionally, the RR for recurrent PE was 0.56 95% CI (0.23 to 1.37).

Conclusions

In patients with intermediate risk PE, the risk of bleeding is higher when thrombolysis is used. There was no significant difference between thrombolysis and anticoagulation in recurrence of PE, stroke, and all‐cause mortality.

Discordant outcomes of laparoscopic versus open appendectomy for suspected appendicitis during pregnancy in published meta-analyses: an overview of systematic reviews

OBJECTIVES

Our study aimed to explore the discrepancy of results between overlapping systematic reviews (SRs) of laparoscopic appendectomy (LA) versus open appendectomy (OA) for suspected appendicitis during pregnancy.

METHODS

MEDLINE, Embase, and Cochrane Database of Systematic Reviews (CDSR) were searched for SRs published from January 1, 2017 to September 10, 2019. SRs and meta-analyses (MA) that compared outcomes of LA versus OA during pregnancy were used. Data regarding the methodology of SR/MA, included studies, efficacy and safety outcomes were extracted. SR quality was analysed with AMSTAR 2.

RESULTS

Four SRs were found, published between April 2018 and April 2019. These reviews included a variety of primary studies, ranging from 17 to 22, and number of included patients varied from 4694 to 6276. A total of 13 outcomes were analyzed. Nine outcomes were included in more than one review; among them, discrepancies between summary effect sizes in meta-analyses were found in four outcomes: preterm birth, Apgar score at 5 min, length of stay in hospital, and wound infection rates. One primary study, which included more than half of the total number of patients in analyzed reviews, showed a predominant effect on the outcome for fetal loss. All four SRs had critically low methodological quality.

CONCLUSIONS

The outcomes of LA versus OA for suspected appendicitis in pregnancy represented in four recent SRs do not provide consistent results. Such uncertainties require new, high-quality primary and secondary evidence on this topic.

Protocol registration issues of systematic review and meta-analysis studies: a survey of global researchers

Background

Although protocol registration of systematic reviews/meta-analysis (SR/MA) is still not mandatory, it is highly recommended that authors publish their SR/MA protocols prior to submitting their manuscripts for publication as recommended by the Cochrane guidelines for conducting SR/MAs. our aim was to assess the awareness, obstacles, and opinions of SR/MA authors about the protocol registration process.

Methods

A cross-sectional survey study included the authors who published SR/MAs during the period from 2010 to 2016, and they were contacted for participation in our survey study. They were identified through the literature search of SR/MAs in Scopus database. An online questionnaire was sent to each participant via e-mail after receiving their approval to join the study. We have sent 6650 emails and received 275 responses.

Results

A total of 270 authors responses were complete and included in the final analysis. Our results has shown that PROSPERO was the most common database used for protocol registration (71.3%). The registration-to-acceptance time interval in PROSPERO was less than 1 month (99.1%). Almost half of the authors (44.2%) did not register their protocols prior to publishing their SR/MAs and according to their opinion that the other authors lack knowledge of protocol importance and mandance to be registered, was the most commonly reported reason (44.9%). A significant percenatge of respondents (37.4%) believed that people would steal their ideas from protocol databases, while only 5.3% reported that their SR/MA had been stolen. However, the majority (72.9%) of participants have agreed that protocol registries play a role in preventing unnecessary duplication of reviews. Finally, 37.4% of participants agree that SR/MA protocol registration should be mandatory.

Conclusion

About half of the participants believes that the main reason for not registering protocols, is that the other authors lack knowledge concerning obligation and importance to register the SR/MA protocols in advance. Therefore, tools should be available to mandate protocol registration of any SRs beforehand and increasing awareness about the benefits of protocol registration among researchers.

Transarterial embolization of acute iatrogenic hemorrhages: predictive factors for mortality and outcome

OBJECTIVE

Iatrogenic hemorrhages occur in 0.5-16% of medical procedures. A retrospective study was conducted to analyze technical and clinical outcome of transarterial embolization (TAE) used for acute iatrogenic hemorrhage and to identify factors predicting outcome.

METHODS

All patients undergoing TAE for acute iatrogenic bleeding from 2006 to 2013 were retrospectively analyzed. Primary end points were technical and clinical success or failure and 30 day mortality.

RESULTS

A total of 153 patients underwent 182 TAEs. Factors associated with clinical failure were lower blood hemoglobin concentration, use of higher number of units of red blood cell concentrate, TAE performed at night or weekend, embolization of more than one vessel, shock state before digital subtraction angiography (DSA), and intensive care before TAE. In multivariate analysis, independent factors for clinical success were hemoglobin concentration, number of units of red blood cell concentrate, and TAE of more than one vessel. Technical failure was associated with female gender, failure to detect signs of bleeding in DSA, TAE of more than one vessel, and shock state before DSA. Bleeding related to anticoagulation medication resulted in a significantly higher mortality rate compared with bleeding due to the remaining causes (30% vs 15%, p < 0.05).

CONCLUSION

Despite excellent technical success, the mortality rate was significant. The only factors affecting clinical success were bleeding intensity and extent of injury. Bleeding attributed to anticoagulation is related to high mortality and therefore requires special attention.

ADVANCES IN KNOWLEDGE

This study gives insights into morbidity and mortality of iatrogenic bleedings and the technical and clinical success rates of TAE in a large study population.

Definition of a systematic review used in overviews of systematic reviews, meta-epidemiological studies and textbooks

BACKGROUND

A standard or consensus definition of a systematic review does not exist. Therefore, if there is no definition about a systematic review in secondary studies that analyse them or the definition is too broad, inappropriate studies might be included in such evidence synthesis. The aim of this study was to analyse the definition of a systematic review (SR) in health care literature, elements of the definitions that are used and to propose a starting point for an explicit and non-ambiguous SR definition.

METHODS

We included overviews of systematic reviews (OSRs), meta-epidemiological studies and epidemiology textbooks. We extracted the definitions of SRs, as well as the inclusion and exclusion criteria that could indicate which definition of a SR the authors used. We extracted individual elements of SR definitions, categorised and quantified them.

RESULTS

Among the 535 analysed sources of information, 188 (35%) provided a definition of a SR. The most commonly used reference points for the definitions of SRs were Cochrane and the PRISMA statement. We found 188 different elements of SR definitions and divided them into 14 categories. The highest number of SR definition elements was found in categories related to searching (N = 51), analysis/synthesis (N = 23), overall methods (N = 22), quality/bias/appraisal/validity (N = 22) and aim/question (N = 13). The same five categories were also the most commonly used combination of categories in the SR definitions.

CONCLUSION

Currently used definitions of SRs are vague and ambiguous, often using terms such as clear, explicit and systematic, without further elaboration. In this manuscript we propose a more specific definition of a systematic review, with the ultimate aim of motivating the research community to establish a clear and unambiguous definition of this type of research.

Submassive Pulmonary Embolism

Pulmonary embolism (PE) presents a spectrum of hemodynamic consequences, ranging from being asymptomatic to a life-threatening medical emergency. Management of submassive and massive PE often involves clinicians from multiple specialties, which can potentially delay the development of a unified treatment plan. In addition, patients with submassive PE can deteriorate after their presentation and require escalation of care. Underlying comorbidities such as chronic obstructive pulmonary disease, cancer, congestive heart failure, and interstitial lung disease can impact the patient's hemodynamic ability to tolerate submassive PE. In this review, we address the definitions, risk stratification (clinical, laboratory, and imaging), management approaches, and long-term outcomes of submassive PE. We also discuss the role of the PE response team in management of patients with PE.

Software engineering principles address current problems in the systematic review ecosystem

Systematic reviewers are simultaneously unable to produce systematic reviews fast enough to keep up with the availability of new trial evidence while overproducing systematic reviews that are unlikely to change practice because they are redundant or biased. Although the transparency and completeness of trial reporting has improved with changes in policy and new technologies, systematic reviews have not yet benefited from the same level of effort. We found that new methods and tools used to automate aspects of systematic review processes have focused on improving the efficiency of individual systematic reviews rather than the efficiency of the entire ecosystem of systematic review production. We use software engineering principles to review challenges and opportunities for improving the interoperability, integrity, efficiency, and maintainability. We conclude by recommending ways to improve access to structured systematic review results. Major opportunities for improving systematic reviews will come from new tools and changes in policy focused on doing the right systematic reviews rather than just doing more of them faster.

The timing and frequency of trial inclusion in systematic reviews of type 2 diabetes drugs was associated with trial characteristics

OBJECTIVE

To determine whether certain trial characteristics are associated with faster or more frequent inclusion in systematic reviews for drug interventions in type 2 diabetes.

STUDY DESIGN AND SETTING

We examined trials included in systematic reviews published between January 1, 2007 and January 1, 2017. Primary outcomes were time between trial publication and first inclusion in a systematic review and frequency of inclusion in systematic reviews over the study period. Multivariable Cox proportional hazards and regression models quantified associations with funding source, number of participants, trial conclusion, and journal impact factor.

RESULTS

Among 668 trials, the median time to inclusion was 76.1 weeks. Time to inclusion was shorter for trials with industry funding (hazard ratio [HR] 1.39; 95% confidence interval [CI] 1.13-1.71), more participants (HR 1.26; 95% CI 1.17-1.36), and published in higher impact factor journals (HR 1.28; 95% CI 1.14-1.45). The median frequency of inclusion was three. Frequency of inclusion was greater for trials with industry funding (relative risk [RR] 2.36; 95% CI 2.11-2.64), more participants (RR 1.51; 95% CI 1.47-1.55), positive conclusions (RR 1.89; 95% CI 1.68-2.13), and published in higher impact factor journals (RR 1.13; 95% CI 1.08-1.18).

CONCLUSION

Certain trial characteristics are associated with faster or more frequent trial inclusion in systematic reviews of type 2 diabetes.

Venous Thromboembolism: Advances in Diagnosis and Treatment

IMPORTANCE

Venous thromboembolism (VTE), comprising deep vein thrombosis (DVT) and pulmonary embolism (PE), is a common and potentially fatal disease.

OBJECTIVE

To summarize the advances in diagnosis and treatment of VTE of the past 5 years.

EVIDENCE REVIEW

A systematic search was conducted in EMBASE Classic, EMBASE, Ovid MEDLINE, and other nonindexed citations using broad terms for diagnosis and treatment of VTE to find systematic reviews and meta-analyses, randomized trials, and prospective cohort studies published between January 1, 2013, and July 31, 2018. The 10th edition of the American College of Chest Physicians Antithrombotic Therapy Guidelines was screened to identify additional studies. Screening of titles, abstracts, and, subsequently, full-text articles was performed in duplicate, as well as data extraction and risk-of-bias assessment of the included articles.

FINDINGS

Thirty-two articles were included in this review. The application of an age-adjusted D-dimer threshold in patients with suspected PE has increased the number of patients in whom imaging can be withheld. The Pulmonary Embolism Rule-Out Criteria safely exclude PE when the pretest probability is low. The introduction of direct oral anticoagulants has allowed for a simplified treatment of VTE with a lower risk of bleeding regardless of etiology or extent of the VTE (except for massive PE) and has made extended secondary prevention more acceptable. Thrombolysis is best reserved for patients with massive PE or those with DVT and threatened limb loss. Insertion of inferior vena cava filters should be avoided unless anticoagulation is absolutely contraindicated in patients with recent acute VTE. Graduated compression stockings are no longer recommended to treat DVT but may be used when acute or chronic symptoms are present. Anticoagulation may no longer be indicated for patients with isolated distal DVT at low risk of recurrence.

CONCLUSIONS AND RELEVANCE

Over the past 5 years, substantial progress has been made in VTE management, allowing for diagnostic and therapeutic strategies tailored to individual patient characteristics, preferences, and values.

Assessing imprecision in Cochrane systematic reviews: a comparison of GRADE and Trial Sequential Analysis

BACKGROUND

The evaluation of imprecision is a key dimension of the grading of the confidence in the estimate. Grading of Recommendations Assessment, Development and Evaluation (GRADE) gives recommendations on how to downgrade evidence for imprecision, but authors vary in their use. Trial Sequential Analysis (TSA) has been advocated for a more reliable assessment of imprecision. We aimed to evaluate reporting of and adherence to GRADE and to compare the assessment of imprecision of intervention effects assessed by GRADE and TSA in Cochrane systematic reviews.

METHODS

In this cross-sectional study, we included 100 Cochrane reviews irrespective of type of intervention with a key dichotomous outcome meta-analyzed and assessed by GRADE. The methods and results sections of each review were assessed for adequacy of imprecision evaluation. We re-analyzed imprecision following the GRADE Handbook and the TSA Manual.

RESULTS

Overall, only 13.0% of reviews stated the criteria they applied to assess imprecision. The most common dimensions were the 95% width of the confidence intervals and the optimal information size. Review authors downgraded 48.0% of key outcomes due to imprecision. When imprecision was re-analyzed following the GRADE Handbook, 64% of outcomes were downgraded. Agreement between review authors' assessment and assessment by the authors of this study was moderate (kappa 0.43, 95% confidence interval [CI] 0.23 to 0.58). TSA downgraded 69.0% outcomes due to imprecision. Agreement between review authors' GRADE assessment and TSA, irrespective of downgrading levels, was moderate (kappa 0.43, 95% CI 0.21 to 0.57). Agreement between our GRADE assessment following the Handbook and TSA was substantial (kappa 0.66, 95% CI 0.49 to 0.79).

CONCLUSIONS

In a sample of Cochrane reviews, methods for assessing imprecision were rarely reported. GRADE according to Handbook guidelines and TSA led to more severe judgment of imprecision rather than GRADE adopted by reviews' authors. Cochrane initiatives to improve adherence to GRADE Handbook are warranted. TSA may transparently assist in such development.

Pulmonary embolism

Pulmonary embolism (PE) is caused by emboli, which have originated from venous thrombi, travelling to and occluding the arteries of the lung. PE is the most dangerous form of venous thromboembolism, and undiagnosed or untreated PE can be fatal. Acute PE is associated with right ventricular dysfunction, which can lead to arrhythmia, haemodynamic collapse and shock. Furthermore, individuals who survive PE can develop post-PE syndrome, which is characterized by chronic thrombotic remains in the pulmonary arteries, persistent right ventricular dysfunction, decreased quality of life and/or chronic functional limitations. Several important improvements have been made in the diagnostic and therapeutic management of acute PE in recent years, such as the introduction of a simplified diagnostic algorithm for suspected PE as well as phase III trials demonstrating the value of direct oral anticoagulants in acute and extended treatment of venous thromboembolism. Future research should aim to address novel treatment options (for example, fibrinolysis enhancers) and improved methods for predicting long-term complications and defining optimal anticoagulant therapy parameters in individual patients, and to gain a greater understanding of post-PE syndrome.