Citation of previous meta-analyses on the same topic: a clue to perpetuation of incorrect methods?

Identification of application and interpretation errors that can occur in pairwise meta-analyses in systematic reviews of interventions: a systematic review

OBJECTIVES

To generate a bank of items describing application and interpretation errors that can arise in pairwise meta-analyses in systematic reviews of interventions.

STUDY DESIGN AND SETTING

MEDLINE, Embase, and Scopus were searched to identify studies describing types of errors in meta-analyses. Descriptions of errors and supporting quotes were extracted by multiple authors. Errors were reviewed at team meetings to determine if they should be excluded, reworded, or combined with other errors, and were categorized into broad categories of errors and subcategories within.

RESULTS

Fifty articles met our inclusion criteria, leading to the identification of 139 errors. We identified 25 errors covering data extraction/manipulation, 74 covering statistical analyses, and 40 covering interpretation. Many of the statistical analysis errors related to the meta-analysis model (eg, using a two-stage strategy to determine whether to select a fixed or random-effects model) and statistical heterogeneity (eg, not undertaking an assessment for statistical heterogeneity).

CONCLUSION

We generated a comprehensive bank of possible errors that can arise in the application and interpretation of meta-analyses in systematic reviews of interventions. This item bank of errors provides the foundation for developing a checklist to help peer reviewers detect statistical errors.

Adhesive strategies in cervical lesions: systematic review and a network meta-analysis of randomized controlled trials

OBJECTIVES

A network meta-analysis (NMA) was performed to assess which adhesive strategy is most clinically effective in treating non-carious cervical lesions (NCCLs).

MATERIAL AND METHODS

Studies were identified by a systematic search of electronic databases including MEDLINE via PubMed, Brazilian Library in Dentistry (BBO), Cochrane Library, EMBASE, Latin American and Caribbean Health Sciences Literature database (LILACS), Scopus, and Web of Science without restrictions on publication year or language. The grey literature was also consulted. Only randomized clinical trials that compared different adhesive strategies in NCCLs in adult patients were included. The risk of bias was evaluated by using the Cochrane Collaboration tool. A random-effects Bayesian mixed treatment comparison model was used to compare adhesive strategies (3ER, 2ER, 2SE, and 1SE) at different follow-up times. The surface under cumulative ranking curve (SUCRA) was estimated for each strategy. Heterogeneity was assessed by using the Cochran Q test and I² statistics. The quality of evidence was evaluated using the GRADE approach.

RESULTS

A total of 5058 studies were identified, 66 of which met the eligibility criteria and of these 5 were judged "low" risk of bias and 57 were meta-analyzed. We did not observe significant differences in the NMA analysis for any two pairs of adhesives, except for the shortest follow-up for 2ER vs 3ER. The material 2SE ranked highest, although it differed only slightly from the other bonding strategies.

CONCLUSIONS

No bonding strategy is better than the others.

CLINICAL RELEVANCE

Adhesive efficacy cannot be characterized by its bonding strategy.

Reliability of the Evidence Addressing Treatment of Corneal Diseases: A Summary of Systematic Reviews

Importance

Patient care should be informed by clinical practice guidelines, which in turn should be informed by evidence from reliable systematic reviews. The American Academy of Ophthalmology is updating its Preferred Practice Patterns (PPPs) for the management of the following 6 corneal diseases: bacterial keratitis, blepharitis, conjunctivitis, corneal ectasia, corneal edema and opacification, and dry eye syndrome.

Objective

To summarize the reliability of the existing systematic reviews addressing interventions for corneal diseases.

Data Source

The Cochrane Eyes and Vision US Satellite database.

Study Selection

In this study of published systematic reviews from 1997 to 2017 (median, 2014), the Cochrane Eyes and Vision US Satellite database was searched for systematic reviews evaluating interventions for the management of any corneal disease, combining eyes and vision keywords and controlled vocabulary terms with a validated search filter.

Data Extraction and Synthesis

The study classified systematic reviews as reliable when each of the following 5 criteria were met: the systematic review specified eligibility criteria for inclusion of studies, conducted a comprehensive literature search for studies, assessed risk of bias of the individual included studies, used appropriate methods for quantitative syntheses (meta-analysis) (only assessed if meta-analysis was performed), and had conclusions that were supported by the results of the systematic review. They were classified as unreliable if at least 1 criterion was not met.

Main Outcomes and Measures

The proportion of systematic reviews that were reliable and the reasons for unreliability.

Results

This study identified 98 systematic reviews that addressed interventions for 15 corneal diseases. Thirty-three of 98 systematic reviews (34%) were classified as unreliable. The most frequent reasons for unreliability were that the systematic review did not conduct a comprehensive literature search for studies (22 of 33 [67%]), did not assess risk of bias of the individual included studies (13 of 33 [39%]), and did not use appropriate methods for quantitative syntheses (meta-analysis) (12 of 17 systematic reviews that conducted a quantitative synthesis [71%]). Sixty-five of 98 systematic reviews (66%) were classified as reliable. Forty-two of the 65 reliable systematic reviews (65%) addressed corneal diseases relevant to the 2018 American Academy of Ophthalmology PPPs; 33 of these 42 systematic reviews (79%) are cited in the 2018 PPPs.

Conclusions and Relevance

One in 3 systematic reviews addressing interventions for corneal diseases are unreliable and thus were not used to inform PPP recommendations. Careful adherence by systematic reviewers and journal editors to well-established best practices regarding systematic review conduct and reporting might help make future systematic reviews in eyes and vision more reliable.

POOLED ESTIMATES OF INCIDENCE OF ENDOPHTHALMITIS AFTER INTRAVITREAL INJECTION OF ANTI-VASCULAR ENDOTHELIAL GROWTH FACTOR AGENTS WITH AND WITHOUT TOPICAL ANTIBIOTIC PROPHYLAXIS

PURPOSE

To assess the effect of topical antibiotic prophylaxis on postoperative endophthalmitis after intravitreal injection of anti-vascular endothelial growth factor agents.

METHODS

A systematic literature search was performed from inception to March 2016 using PubMed, Medline, Web of Science, Embase, and the Cochrane Library, to identify articles that reported cases of endophthalmitis after intravitreal injection of anti-vascular endothelial growth factor agents. We used a pooled analysis to estimate the incidence of cases of endophthalmitis who developed after injections performed with and without topical antibiotic prophylaxis. We used regression analysis to explore the effects of study characteristics on heterogeneity.

RESULTS

From our search of electronic databases, we identified and screened 4,561 unique records. We judged 60 articles to have reported findings for cohorts of patients who met our inclusion criteria, (12 arms of randomized clinical trials, 11 prospective cohort studies, and 37 retrospective cohort studies), which included 244 cases of endophthalmitis and 639,391 intravitreal injections of anti-vascular endothelial growth factor agents. The final pooled estimate endophthalmitis proportions were 9/10,000 (95% confidence interval, 7/10,000-12/10,000) in the antibiotic-treated group and 3/10,000 (95% confidence interval, 2/10,000-5/10,000) in the untreated group. The estimated incidence of endophthalmitis with topical antibiotic prophylaxis was approximated three times the incidence without prophylaxis. Random effects regression showed that none of the study characteristics significantly affected the effect size in either group.

CONCLUSION

Topical antibiotic after intravitreal injection of anti-vascular endothelial growth factor agents is associated with a higher risk of endophthalmitis.

Network Meta-analysis for Clinical Practice Guidelines: A Case Study on First-Line Medical Therapies for Primary Open-Angle Glaucoma

BACKGROUND

Network meta-analysis compares multiple treatment options for the same condition and may be useful for developing clinical practice guidelines.

PURPOSE

To compare treatment recommendations for first-line medical therapy for primary open angle-glaucoma (POAG) from major updates of American Academy of Ophthalmology (AAO) guidelines with the evidence available at the time, using network meta-analysis.

DATA SOURCES

MEDLINE, Embase, and the Cochrane Library were searched on 11 March 2014 for randomized, controlled trials (RCTs) of glaucoma monotherapies compared with placebo, vehicle, or no treatment or other monotherapies. The AAO Web site was searched in August 2014 to identify AAO POAG guidelines.

STUDY SELECTION

Eligible RCTs were selected by 2 independent reviewers, and guidelines were selected by 1 person.

DATA EXTRACTION

One person abstracted recommendations from guidelines and a second person verified. Two people independently abstracted data from included RCTs.

DATA SYNTHESIS

Guidelines were grouped together on the basis of literature search dates, and RCTs that existed at 1991, 1995, 1999, 2004, and 2009 were analyzed. The outcome of interest was intraocular pressure (IOP) at 3 months. Only the latest guideline made a specific recommendation: prostaglandins. Network meta-analyses showed that all treatments were superior to placebo in decreasing IOP at 3 months. The mean reductions (95% credible intervals [CrIs]) for the highest-ranking class compared with placebo were as follows: 1991: β-blockers, 4.01 (CrI, 0.48 to 7.43); 1995: α2-adrenergic agonists, 5.64 (CrI, 1.73 to 9.50); 1999: prostaglandins, 5.43 (CrI, 3.38 to 7.38); 2004: prostaglandins, 4.75 (CrI, 3.11 to 6.44); 2009: prostaglandins, 4.58 (CrI, 2.94 to 6.24).

LIMITATION

When comparisons are informed by a small number of studies, the treatment effects and rankings may not be stable.

CONCLUSION

For timely recommendations when multiple treatment options are available, guidelines developers should consider network meta-analysis.

PRIMARY FUNDING SOURCE

National Eye Institute, National Institutes of Health.

Interventions for Age-Related Macular Degeneration: Are Practice Guidelines Based on Systematic Reviews?

PURPOSE

Are existing systematic reviews of interventions for age-related macular degeneration incorporated into clinical practice guidelines?

DESIGN

High-quality systematic reviews should be used to underpin evidence-based clinical practice guidelines and clinical care. We examined the reliability of systematic reviews of interventions for age-related macular degeneration (AMD) and described the main findings of reliable reviews in relation to clinical practice guidelines.

METHODS

Eligible publications were systematic reviews of the effectiveness of treatment interventions for AMD. We searched a database of systematic reviews in eyes and vision without language or date restrictions; the database was up to date as of May 6, 2014. Two authors independently screened records for eligibility and abstracted and assessed the characteristics and methods of each review. We classified reviews as reliable when they reported eligibility criteria, comprehensive searches, methodologic quality of included studies, appropriate statistical methods for meta-analysis, and conclusions based on results. We mapped treatment recommendations from the American Academy of Ophthalmology (AAO) Preferred Practice Patterns (PPPs) for AMD to systematic reviews and citations of reliable systematic reviews to support each treatment recommendation.

RESULTS

Of 1570 systematic reviews in our database, 47 met inclusion criteria; most targeted neovascular AMD and investigated anti-vascular endothelial growth factor (VEGF) interventions, dietary supplements, or photodynamic therapy. We classified 33 (70%) reviews as reliable. The quality of reporting varied, with criteria for reliable reporting met more often by Cochrane reviews and reviews whose authors disclosed conflicts of interest. Anti-VEGF agents and photodynamic therapy were the only interventions identified as effective by reliable reviews. Of 35 treatment recommendations extracted from the PPPs, 15 could have been supported with reliable systematic reviews; however, only 1 recommendation cited a reliable intervention systematic review. No reliable systematic review was identified for 20 treatment recommendations, highlighting areas of evidence gaps.

CONCLUSIONS

For AMD, reliable systematic reviews exist for many treatment recommendations in the AAO PPPs and should be cited to support these recommendations. We also identified areas where no high-level evidence exists. Mapping clinical practice guidelines to existing systematic reviews is one way to highlight areas where evidence generation or evidence synthesis is either available or needed.

Comparative Effectiveness of First-Line Medications for Primary Open-Angle Glaucoma: A Systematic Review and Network Meta-analysis

TOPIC

Primary open-angle glaucoma (POAG) is a highly prevalent condition worldwide and the most common cause of irreversible sight loss. The objective is to assess the comparative effectiveness of first-line medical treatments in patients with POAG or ocular hypertension through a systematic review and network meta-analysis, and to provide relative rankings of these treatments.

CLINICAL RELEVANCE

Treatment for POAG currently relies completely on lowering the intraocular pressure (IOP). Although topical drops, lasers, and surgeries can be considered in the initial treatment of glaucoma, most patients elect to start treatment with eye drops.

METHODS

We included randomized controlled trials (RCTs) that compared a single active topical medication with no treatment/placebo or another single topical medication. We searched CENTRAL, MEDLINE, EMBASE, and the Food and Drug Administration's website. Two individuals independently assessed trial eligibility, abstracted data, and assessed the risk of bias. We performed Bayesian network meta-analyses.

RESULTS

We included 114 RCTs with data from 20 275 participants. The overall risk of bias of the included trials is mixed. The mean reductions (95% credible intervals) in IOP in millimeters of mercury at 3 months ordered from the most to least effective drugs were as follows: bimatoprost 5.61 (4.94; 6.29), latanoprost 4.85 (4.24; 5.46), travoprost 4.83 (4.12; 5.54), levobunolol 4.51 (3.85; 5.24), tafluprost 4.37 (2.94; 5.83), timolol 3.70 (3.16; 4.24), brimonidine 3.59 (2.89; 4.29), carteolol 3.44 (2.42; 4.46), levobetaxolol 2.56 (1.52; 3.62), apraclonidine 2.52 (0.94; 4.11), dorzolamide 2.49 (1.85; 3.13), brinzolamide 2.42 (1.62; 3.23), betaxolol 2.24 (1.59; 2.88), and unoprostone 1.91 (1.15; 2.67).

CONCLUSIONS

All active first-line drugs are effective compared with placebo in reducing IOP at 3 months. Bimatoprost, latanoprost, and travoprost are among the most efficacious drugs, although the within-class differences were small and may not be clinically meaningful. All factors, including adverse effects, patient preferences, and cost, should be considered in selecting a drug for a given patient.

Global reported endophthalmitis risk following intravitreal injections of anti-VEGF: a literature review and analysis

Purpose

To report on endophthalmitis occurrence and associated risk factors following the intravitreal injection of anti-VEGF agents based on a review of published literature.

Materials and methods

A Medline search was performed using the terms “bevacizumab” and “ranibizumab”. A total of 534 English-language articles of varying design and published from 2006 to November 2013 were analyzed for endophthalmitis occurrence and contributing perioperative factors.

Results

A total of 445,503 injections were counted. There were 103 cases of postinjection endophthalmitis in 176,124 injections (0.058%) with bevacizumab (Avastin) versus 79 cases in 269,379 injections (0.029%) with ranibizumab (Lucentis). This difference was due to a significantly higher occurrence of culture-negative endophthalmitis associated with bevacizumab injections. Culture-positive risk was not statistically different between the two drugs. The reported use of postinjection topical antibiotics increased the risk of culture-positive endophthalmitis. No association was found with the use of povidone iodine, a lid speculum, a mask, or an operating room. Streptococcus spp. were the most prevalent causative organism, accounting for nine of 54 (17%) of all culture-positive cases.

Conclusion

Reported postinjection endophthalmitis occurred significantly more in patients treated with bevacizumab than those treated with ranibizumab. However, culture-positive occurrence was similar. Despite the potential for contamination at the time of drug compounding, bevacizumab does not appear to confer a higher risk of culture-positive endophthalmitis than ranibizumab. This study also suggests antibiotic use may increase endophthalmitis occurrence.

Efficacy and tolerability of mono-compound topical treatments for reduction of intraocular pressure in patients with primary open angle glaucoma or ocular hypertension: an overview of reviews

AIM

To evaluate the existing evidence on relative efficacy and tolerability of topical mono-compound intraocular pressure (IOP)-lowering drugs in treatment of primary open angle glaucoma (POAG) and ocular hypertension (OHT).

METHODS

In this systematic review of systematic reviews/meta-analyses of randomized controlled trials a thorough and sensitive search of PubMed, Embase and Cochrane Databases was performed. Individual study methodological quality and quality of evidence were assessed using the AMSTAR checklist and the GRADE system, respectively. The relationships between individual drugs were evaluated based on the best available evidence.

RESULTS

Of the 133 initial non-duplicate records, 16 studies met the inclusion criteria. Five achieved an overall "moderate" (none achieved "high") quality of evidence and evaluated prostaglandin analogues (PGAs) - latanoprost, travoprost, and bimatoprost; timolol; "other beta-blockers;" carbonic anhydrase inhibitors (CAI) as a group or dorzolamide separately; and brimonidine. "Moderate quality" refers to efficacy and incidence of conjunctival hyperemia. Quality of evidence regarding other tolerability aspects was low. PGAs should be considered equivalent regarding efficacy, but latanoprost was relevantly better tolerated than the other two. Non-PGA compounds did not relevantly differ between each other in either efficacy or safety. Timolol and brimonidine were relevantly less effective than all PGAs. The same was true for CAI vs bimatoprost. Regarding tolerability, timolol was superior to all PGAs and brimonidine and CAI were superior to bimatoprost.

CONCLUSION

No high quality evidence on relative efficacy and tolerability of the most commonly used mono-compound IOP-lowering drugs for POAG/OHT exists. Moderate quality evidence indicates latanoprost as a treatment with the most favorable trade-off between benefits and harms.

Learning by doing-teaching systematic review methods in 8 weeks

OBJECTIVE

The objective of this paper is to describe the course "Systematic Reviews and Meta-analysis" at the Johns Hopkins Bloomberg School of Public Health.

METHODS

A distinct feature of our course is a group project in which students, assigned to multi-disciplinary groups, conduct a systematic review. In-class sessions comprise didactic lectures, hands-on exercises, demonstrations, discussion, and group work. Students also work outside of class to complete the systematic review. Students evaluated the course at the end of the term. We also surveyed students from 2004 to 2012 to learn more about the long-term impact of the course.

RESULTS

The course has been offered to more than 800 students since 1995. In our view, aspects that worked well include the hands-on approach, students working in a multidisciplinary group, intensive interaction with the teaching team, moving to an online approach, and continuous updates of the course content. A persistent issue is the constraint of time. 193 of 211 (91%) survey participants reported that the course is currently useful or as having an impact on their work.

CONCLUSIONS

Our experiences have led us to remain committed to a hands-on approach. Our course serves as a bridge between classroom learning and real-world practice, and provides an example of teaching systematic review.

Meta-analysis of randomized controlled trials comparing latanoprost with other glaucoma medications in chronic angle-closure glaucoma

PURPOSE

To evaluate the efficacy and safety of latanoprost compared with other glaucoma medications in the treatment of chronic angle-closure glaucoma (CACG) and to provide the basis for clinical medication.

METHODS

Major literature databases were searched for randomized controlled trials (RCT) involving latanoprost among patients with CACG. Primary outcome measures were absolute changes in intraocular pressure (IOP) and incidence of ocular adverse events. Statistical analyses included the calculation of standardized mean difference (SMD) and relative risk (RR). The statistical analysis was performed using STATA version 12.0 software.

RESULTS

Ten RCT involving 1096 patients were included in this meta-analysis. Analysis showed that latanoprost was not significantly different from other glaucoma medications in reducing IOP (SMD = 0.29, 95% confidence interval [CI] -0.02 to 0.59, p=0.069). Further subgroup analysis revealed that latanoprost was superior compared with timolol (SMD = 0.64, 95% CI 0.46 to 0.82, p<0.001) and marginally inferior to travoprost and bimatoprost (SMD = -0.19, 95% CI -0.35 to -0.02, p = 0.026). As for conjunctival hyperemia, latanoprost caused a higher proportion than timolol (RR = 2.36, 95% CI 1.27 to 4.37, p = 0.007). However, latanoprost was associated with lower incidence of conjunctival hyperemia (RR = 0.42, 95% CI 0.30 to 0.59, p<0.001), and with fewer occurrence of other ocular side effects (excluding conjunctival hyperemia) than travoprost and bimatoprost (RR = 0.61, 95% CI 0.48 to 0.78, p<0.001).

CONCLUSIONS

Travoprost and bimatoprost are superior in IOP control than latanoprost, but latanoprost is better tolerated in patients with CACG.

Meta-analysis of randomized controlled trials comparing EX-PRESS implantation with trabeculectomy for open-angle glaucoma

Purpose

To evaluate the efficacy and safety of EX-PRESS implantation compared with trabeculectomy for uncontrolled open-angle glaucoma.

Methods

Pertinent randomized controlled trials were identified through systematic searches of the PubMed, EMBASE, and Cochrane Library. The efficacy measures utilized were the weighted mean differences (WMDs) for the intraocular pressure reduction (IOPR), the reduction in glaucoma medications, the change of visual acuity, and the relative risks (RRs) for operative success rates. The safety measures utilized were RRs for postoperative complications. The pooled effects were calculated using the random-effects model.

Results

Four randomized controlled trials of 292 eyes were included in this meta-analysis. The WMDs of the IOPR comparing the EX-PRESS with trabeculectomy were −0.25 (95% Cl: −3.61 to 3.11) at 6 month, 0.053 (−4.31 to 4.42) at 12 months, 0.81 (−4.06 to 5.67) at 24 months, and 0.20 (−2.11 to 2.51) at final follow-up. There was no statistically significance for IOPR at any point after surgery. There were also no significant differences in the reduction in glaucoma medications or visual acuity between the groups. The pooled relative risks comparing EX-PRESS with Trabeculectomy were 1.36 (1.11 to 1.66) for the complete operative success rate and 1.05 (0.94 to 1.17) for the qualified operative success rate. EX-PRESS and Trabeculectomy were associated with similar incidences in most complications with the exception of hyphema, with pooled RR being 0.18 (0.046 to 0.66).

Conclusions

EX-PRESS implantation and trabeculectomy have similar efficacy in IOP-lowering, medication reduction, vision recovery, and qualified operative success rates. EX-PRESS associated with higher rates of complete operative success and fewer hyphema than with Trabeculectomy. However, these should be interpreted with caution because of the inherent limitations of the included studies.

Efficacy and tolerability of the fixed combinations latanoprost/timolol versus dorzolamide/timolol in patients with elevated intraocular pressure: a meta-analysis of randomized controlled trials

Objective

To evaluate the efficacy and tolerability of the fixed combination of Latanoprost/Timolol versus Dorzolamide/Timolol in the treatment of patients with elevated intraocular pressure (IOP).

Methods

A comprehensive literature meta-analysis was performed according to the Cochrane Collaboration methodology to identify randomized clinical trials comparing latanoprost/timolol FC (FCLT) with dorzolamide/timolol (FCDT) in patients with elevated IOP. The efficacy estimates were measured by the weight mean difference (WMD) for the IOP reduction (IOPR) from baseline to end point, including the diurnal mean IOPR, 8 AM IOPR, 12 PM IOPR, and 4 PM IOPR. The tolerability estimates were measured by RR for adverse events. All outcomes were reported with a 95% confidence interval (CI). The data were synthesized by Stata 12.0 SE for Windows.

Results

Eight studies involving 841 patients (841 eyes) were included in the meta-analysis. With a WMD of IOPR in the diurnal mean of 0.16 mmHg (95% CI, -0.31 to 0.63), the FCLT was as effective as FCDT in lowering IOP in patients with elevated IOP (P = 0.51). The WMDs of IOPR were 0.58 mmHg (95% CI: -0.002 to 1.17) at 8 AM, -0.07 mmHg (95% CI: -0.50 to 0.36) at 12 PM, and 0.41 mmHg (95% CI: -0.18 to 1.00) at 4 PM, and there were no significant difference between FCLT and FCDT. FCLT was associated with a significantly lower incidence of eye pain, bitter taste, and irritation/stinging than FCDT, with pooled RRs of 0.34 (95% CI: 0.14 to 0.82), 0.06 (95% CI:0.008 to 0.42), and 0.35 (95% CI: 0.14 to 0.85), respectively.

Conclusion

FCLT was associated with equivalent efficacy in IOP lowering comparing with FCDT. However, FCLT was better tolerated than FCDT.