The quality of systematic reviews about interventions for refractive error can be improved: a review of systematic reviews

Guidance to best tools and practices for systematic reviews

Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy. A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work. Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

Interventions for Dry Eye: An Overview of Systematic Reviews

Importance

Dry eye is a common ocular disease that can have substantial morbidity. Systematic reviews provide evidence for dry eye interventions and can be useful for patients, clinicians, and clinical guideline developers. Overviews of reviews use explicit and systematic methods to synthesize findings from multiple systematic reviews, but currently, there are no overviews of systematic reviews investigating interventions for dry eye.

Objective

To summarize the results of reliable systematic reviews of dry eye interventions and to highlight the evidence gaps identified.

Evidence Review

We searched the Cochrane Eyes and Vision US satellite database and included reliable systematic reviews evaluating dry eye interventions published from 2016 to 2022. We reported the proportion of systematic reviews that were reliable with reasons for unreliability. Critical and important outcomes from reliable systematic reviews were extracted and verified. Critical outcomes included dry eye-related patient-reported outcome measures. Results were synthesized from reliable systematic reviews to provide summaries of evidence for each intervention. Evidence for each intervention was defined as conclusive or inconclusive depending on whether high-certainty evidence across systematic reviews was available according to Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) criteria and whether findings reached statistical or clinical significance. Recommendations were made for further research.

Findings

Within the Cochrane Eyes and Vision US satellite database, 138 potentially relevant systematic reviews were identified, 71 were considered eligible, and 26 (37%) were assessed as reliable. Among reliable systematic reviews, no conclusive evidence was identified for any dry eye intervention. Inconclusive evidence suggested that environmental modifications, dietary modifications, artificial tears and lubricants, punctal occlusion, intense pulsed light therapy, vectored thermal pulsation therapy (Lipiflow), topical corticosteroids, topical cyclosporine A, topical secretagogues, and autologous serum may be effective. Only unreliable systematic reviews evaluated lifitegrast, oral antibiotics, and moisture chamber devices.

Conclusions and Relevance

This overview of systematic reviews found some evidence that dry eye interventions may be effective, but no conclusive evidence was available. The conduct and reporting of most systematic reviews for dry eye interventions warrant improvement, and reliable systematic reviews are needed to evaluate lifitegrast, oral antibiotics, and moisture chamber devices.

Ultrathin DSAEK versus DMEK - Review of systematic reviews

The efficacy and safety of Descemet's membrane endothelial keratoplasty (DMEK) and ultrathin Descemet stripping automated endothelial keratoplasty (UT-DSAEK) have been recently compared in several systematic reviews (SRs). The aim of this study was to assess the evidence quality of such SRs, in order to obtain a scientifically rigorous comparison between the two techniques. We performed a systematic review of SRs and meta-analyses comparing the efficacy and safety between UT-DSAEK and DMEK up to 24th March 2023, using 3 electronic databases (PubMed, Cochrane Library, Google Scholar) plus manual reference search. Specific outcomes analyzed included best-corrected visual acuity (BCVA), endothelial cell density (ECD), rebubbling rate, and other postoperative complications. Of 90 titles/abstracts screened, four SRs met the inclusion criteria. All SRs adequately analyzed potential bias of the included studies. One SR raised concern for potential literature search bias and two SRs have heterogeneity in some outcomes analyzed. All SRs found higher BCVA after DMEK, but one SR reported significant heterogeneity. All SRs found significant heterogeneity in ECD analysis, with one SR providing inconsistent analysis of this outcome. Three SRs analyzed rebubbling rates, favoring UT-DSAEK over DMEK. Three SRs concluded a higher overall complication rate after DMEK, although rebubbling may be a confounding factor. This systematic review clarifies the strengths and weaknesses of published SRs and reinforces the conclusion that DMEK leads to superior visual outcomes compared to UT-DSAEK, with the trade-off of higher rebubbling rates and possibly other postoperative complications. Studies with longer follow-up are needed to ascertain these differences between procedures.

Guidance to best tools and practices for systematic reviews

Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy. A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work. Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

Guidance to best tools and practices for systematic reviews

Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy. A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work. Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

Guidance to best tools and practices for systematic reviews

Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy.A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work.Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

Guidance to best tools and practices for systematic reviews

Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy.A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work.Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

Guidance to Best Tools and Practices for Systematic Reviews

» Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy.» A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work.» Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

TFOS Lifestyle - Evidence quality report: Advancing the evaluation and synthesis of research evidence

Evidence-based practice is a dominant paradigm in healthcare that emphasizes the importance of ensuring the translation of the best available, relevant research evidence into practice. An Evidence Quality Subcommittee was established to provide specialized methodological support and expertise to promote rigorous and evidence-based approaches for the Tear Film and Ocular Surface Society (TFOS) Lifestyle Epidemic reports. The present report describes the purpose, scope, and activity of the Evidence Quality Subcommittee in the undertaking of high-quality narrative-style literature reviews, and leading prospectively registered, reliable systematic reviews of high priority research questions, using standardized methods for each topic area report. Identification of predominantly low or very low certainty evidence across the eight systematic reviews highlights a need for further research to define the efficacy and/or safety of specific lifestyle interventions on the ocular surface, and to clarify relationships between certain lifestyle factors and ocular surface disease. To support the citation of reliable systematic review evidence in the narrative review sections of each report, the Evidence Quality Subcommittee curated topic-specific systematic review databases and relevant systematic reviews underwent standardized reliability assessment. Inconsistent methodological rigor was noted in the published systematic review literature, emphasizing the importance of internal validity assessment. Based on the experience of implementing the Evidence Quality Subcommittee, this report makes suggestions for incorporation of such initiatives in future international taskforces and working groups. Content areas broadly relevant to the activity of the Evidence Quality Subcommittee, including the critical appraisal of research, clinical evidence hierarchies (levels of evidence), and risk of bias assessment, are also outlined.

Guidance to best tools and practices for systematic reviews1

Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy.A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work.Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

Randomized Controlled Trials of Digital Mental Health Interventions on Patients with Schizophrenia Spectrum Disorder: A Systematic Review

Background: This systematic review aimed to examine the study protocol of Digital Mental Health Interventions (DMHIs) and to review the effect of DMHIs among patients with Schizophrenia Spectrum Disorder (SSD). Methods: This review followed the guideline of Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). A systematic literature search was performed using PubMed, CINAHL, Embase, and PsycINFO electronic databases to identify randomized clinical trials without any limit on the publication year. Overall, 18 studies were selected and evaluated for the quality assessment utilizing the Risk of Bias 2 tool of Cochranes' Collaboration. In the quality assessment, four studies evaluated as overall high risk of bias were excluded from the selection, and the final 14 studies were chosen. Results: No DMHIs were provided for acute schizophrenia-related symptoms, and there were some studies related to schizophrenia-related symptoms (26.4%). Some studies for improving cognitive function (42.9%) were reported, and there was a significant effect when interventions that were proven to be effective when implemented in a face-to-face manner were delivered using various online devices and sensory stimuli. Nearly half of the studies reported intervention frequency and time (57.1%), and those with unclear reports relied either on a mobile app or telemedicine and were designed to self-pace the frequency and speed of the intervention. Conclusion: Based on our findings, it will be possible to understand the characteristics of DMHIs, without physical contact, for only SSD patients, providing a basis for digital mental health services.

Effectiveness of interventions for dry eye: a protocol for an overview of systematic reviews

INTRODUCTION

Dry eye is a leading cause of ocular morbidity and economic and societal burden for patients and healthcare systems. There are several treatment options available for dry eye and high-quality systematic reviews synthesise the evidence for their effectiveness and potential harms.

METHODS AND ANALYSIS

We will search the Cochrane Eyes and Vision US satellite (CEV@US) database of eyes and vision systematic reviews for systematic reviews on interventions for dry eye. CEV@US conducted an initial search of PubMed and Embase to populate the CEV@US database of eyes and vision systematic reviews in 2007, which was updated most recently in August 2021. We will search the database for systematic reviews published since 1 January 2016 because systematic reviews more than 5 years are unlikely to be up to date. We will consider Cochrane and non-Cochrane systematic reviews eligible for inclusion. Two authors will independently screen articles. We will include studies that evaluate interventions for dry eye and/or meibomian gland dysfunction with no restriction on types of participants or review language. We will select reliable systematic reviews (ie, those meeting pre-established methodological criteria) for inclusion, assessed by one investigator and verified by a second investigator. We will extract ratings of the certainty of evidence from within each review. We will report the degree of overlap for systematic reviews that answer similar questions and include overlapping primary studies. We will present results of the overview in alignment with guidelines in the Cochrane Handbook of Systematic Reviews of Interventions (Online Chapter 5: Overviews of Reviews), the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement, and an overview of reviews quality and transparency checklist. The anticipated start and completion dates for this overview are 1 May 2021 and 30 April 2022, respectively.

ETHICS AND DISSEMINATION

This overview will not require the approval of an Ethics Committee because it will use published studies. We will publish results in a peer-reviewed journal.

PROSPERO REGISTRATION NUMBER

CRD42021279880.

Harms in Systematic Reviews Paper 2: Methods used to assess harms are neglected in systematic reviews of gabapentin

OBJECTIVE

We compared methods used with current recommendations for synthesizing harms in systematic reviews and meta-analyses (SRMAs) of gabapentin.

STUDY DESIGN & SETTING

We followed recommended systematic review practices. We selected reliable SRMAs of gabapentin (i.e., met a pre-defined list of methodological criteria) that assessed at least one harm. We extracted and compared methods in four areas: pre-specification, searching, analysis, and reporting. Whereas our focus in this paper is on the methods used, Part 2 examines the results for harms across reviews.

RESULTS

We screened 4320 records and identified 157 SRMAs of gabapentin, 70 of which were reliable. Most reliable reviews (51/70; 73%) reported following a general guideline for SRMA conduct or reporting, but none reported following recommendations specifically for synthesizing harms. Across all domains assessed, review methods were designed to address questions of benefit and rarely included the additional methods that are recommended for evaluating harms.

CONCLUSION

Approaches to assessing harms in SRMAs we examined are tokenistic and unlikely to produce valid summaries of harms to guide decisions. A paradigm shift is needed. At a minimal, reviewers should describe any limitations to their assessment of harms and provide clearer descriptions of methods for synthesizing harms.

Quality appraisal of systematic reviews of interventions for children with cerebral palsy reveals critically low confidence

AIM

To evaluate the methodological quality of recent systematic reviews of interventions for children with cerebral palsy in order to determine the level of confidence in the reviews' conclusions.

METHOD

A comprehensive search of 22 databases identified eligible systematic reviews with and without meta-analysis published worldwide from 2015 to 2019. We independently extracted data and used A Measurement Tool to Assess Systematic Reviews-2 (AMSTAR-2) to appraise methodological quality.

RESULTS

Eighty-three systematic reviews met strict eligibility criteria. Most were from Europe and Latin America and reported on rehabilitative interventions. AMSTAR-2 appraisal found critically low confidence in 88% (n=73) because of multiple and varied deficiencies. Only 7% (n=6) had no AMSTAR-2 critical domain deficiency. The number of systematic reviews increased fivefold from 2015 to 2019; however, quality did not improve over time.

INTERPRETATION

Most of these systematic reviews are considered unreliable according to AMSTAR-2. Current recommendations for treating children with CP based on these flawed systematic reviews need re-evaluation. Findings are comparable to reports from other areas of medicine, despite the general perception that systematic reviews are high-level evidence. The required use of current widely accepted guidance for conducting and reporting systematic reviews by authors, peer reviewers, and editors is critical to ensure reliable, unbiased, and transparent systematic reviews. What this paper adds Confidence was critically low in the conclusions of 88% of systematic reviews about interventions for children with cerebral palsy (CP). Quality issues in the sample were not limited to systematic reviews of non-randomized trials, or to those about certain populations of CP or interventions. The inclusion of meta-analysis did not improve the level of confidence in these systematic reviews. Numbers of systematic reviews on this topic increased over the 5 search years but their methodological quality did not improve.

What Do We Really Know about the Effectiveness of Glaucoma Interventions?: An Overview of Systematic Reviews

PURPOSE

To identify systematic reviews of interventions for glaucoma conditions and to assess their reliability, thereby generating a list of potentially reliable reviews for updating glaucoma practice guidelines.

DESIGN

Cross-sectional study.

PARTICIPANTS

Systematic reviews of interventions for glaucoma conditions.

METHODS

We used a database of systematic reviews and meta-analyses in vision research and eye care maintained by the Cochrane Eyes and Vision United States Satellite. We examined all Cochrane systematic reviews of interventions for glaucoma conditions published before August 7, 2019, and all non-Cochrane systematic reviews of interventions for glaucoma conditions published between January 1, 2014, and August 7, 2019.

MAIN OUTCOME MEASURES

We assessed eligible reviews for reliability, extracted characteristics, and summarized key findings from reviews classified as reliable.

RESULTS

Of the 4451 systematic reviews in eyes and vision identified, 129 met our eligibility criteria and were assessed for reliability. Of these, we classified 49 (38%) as reliable. We found open-angle glaucoma (22/49) to be the condition with the most reviews and medical management (17/49) and intraocular pressure (IOP; 43/49) to be the most common interventions and outcomes studied. Most reviews found a high degree of uncertainty in the evidence, which hinders the possibility of making strong recommendations in guidelines. These reviews found high-certainty evidence about a few topics: reducing IOP helps to prevent glaucoma and its progression, prostaglandin analogs are the most effective medical treatment for lowering IOP, laser trabeculoplasty is as effective as medical treatment as a first-line therapy in controlling IOP, the use of IOP-lowering medications in the perioperative or postoperative periods to accompany laser (e.g., trabeculoplasty) reduces the risk of postoperative IOP spikes, conventional surgery (i.e., trabeculectomy) is more effective than medications in reducing IOP, and antimetabolites and β-radiation improve IOP control after trabeculectomy. The evidence is weak regarding the effectiveness of minimally invasive glaucoma surgeries.

CONCLUSIONS

Most systematic reviews evaluating interventions for glaucoma are of poor reliability. Even among those that may be considered reliable, important limitations exist in the value of information because of the uncertainty of the evidence as well as small and sometimes unimportant clinical differences between interventions.

Evaluation of Systematic Reviews of Interventions for Retina and Vitreous Conditions

Importance

Patient care and clinical practice guidelines should be informed by evidence from reliable systematic reviews. The reliability of systematic reviews related to forthcoming guidelines for retina and vitreous conditions is unknown.

Objectives

To summarize the reliability of systematic reviews on interventions for 7 retina and vitreous conditions, describe characteristics of reliable and unreliable systematic reviews, and examine the primary area in which they appeared to be lacking.

Design, Setting, and Participants

A cross-sectional study of systematic reviews was conducted. Systematic reviews of interventions for retina- and vitreous-related conditions in a database maintained by the Cochrane Eyes and Vision United States Satellite were identified. Databases that the reviewers searched, whether any date or language restrictions were applied, and bibliographic information, such as year and journal of publication, were documented. The initial search was conducted in March 2007, and the final update was performed in July 2018. The conditions of interest were age-related macular degeneration; diabetic retinopathy; idiopathic epiretinal membrane and vitreomacular traction; idiopathic macular hole; posterior vitreous detachment, retinal breaks, and lattice degeneration; retinal and ophthalmic artery occlusions; and retinal vein occlusions. The reliability of each review was evaluated using prespecified criteria. Data were extracted by 2 research assistants working independently, with disagreements resolved through discussion or by 1 research assistant with verification by a senior team member.

Main Outcomes and Measures

Proportion of reviews that meet all of the following criteria: (1) defined eligibility criteria for study selection, (2) described conducting a comprehensive literature search, (3) reported assessing risk of bias in included studies, (4) described using appropriate methods for any meta-analysis performed, and (5) provided conclusions consistent with review findings.

Results

A total of 327 systematic reviews that addressed retina and vitreous conditions were identified; of these, 131 reviews (40.1%) were classified as reliable and 196 reviews (59.9%) were classified as not reliable. At least 1 reliable review was found for each of the 7 retina and vitreous conditions. The most common reason that a review was classified as not reliable was lack of evidence that a comprehensive literature search for relevant studies had been conducted (149 of 196 reviews [76.0%]).

Conclusion and Relevance

The findings of this study suggest that most systematic reviews that addressed interventions for retina and vitreous conditions were not reliable. Systematic review teams and guideline developers should work with information professionals who can help navigate sophisticated and varied syntaxes required to search different resources.

Accuracy of optical coherence tomography for diagnosing glaucoma: an overview of systematic reviews

AIMS

To assess the diagnostic accuracy (DTA) of optical coherence tomography (OCT) for detecting glaucoma by systematically searching and appraising systematic reviews (SRs) on this issue.

METHODS

We searched a database of SRs in eyes and vision maintained by the Cochrane Eyes and Vision United States on the DTA of OCT for detecting glaucoma. Two authors working independently screened the records, abstracted data and assessed the risk of bias using the Risk of Bias in Systematic Reviews checklist. We extracted quantitative DTA estimates as well as qualitative statements on their relevance to practice.

RESULTS

We included four SRs published between 2015 and 2018. These SRs included between 17 and 113 studies on OCT for glaucoma diagnosis. Two reviews were at low risk of bias and the other two had two to four domains at high or unclear risk of bias with concerns on applicability. The two reliable SRs reported the accuracy of average retinal nerve fibre layer (RNFL) thickness and found a sensitivity of 0.69 (0.63 to 0.73) and 0.78 (0.74 to 0.83) and a specificity of 0.94 (0.93 to 0.95) and 0.93 (0.92 to 0.95) in 57 and 50 studies, respectively. Only one review included a clear specification of the clinical pathway. Both reviews highlighted the limitations of primary DTA studies on this topic.

CONCLUSIONS

The quality of published DTA reviews on OCT for diagnosing glaucoma was mixed. Two reliable SRs found moderate sensitivity at high specificity for average RNFL thickness in diagnosing manifest glaucoma. Our overview suggests that the methodological quality of both primary and secondary DTA research on glaucoma is in need of improvement.

Evaluating the quality of evidence for gaming disorder: A summary of systematic reviews of associations between gaming disorder and depression or anxiety

Gaming disorder has been described as an urgent public health problem and has garnered many systematic reviews of its associations with other health conditions. However, review methodology can contribute to bias in the conclusions, leading to research, policy, and patient care that are not truly evidence-based. This study followed a pre-registered protocol (PROSPERO 2018 CRD42018090651) with the objective of identifying reliable and methodologically-rigorous systematic reviews that examine the associations between gaming disorder and depression or anxiety in any population. We searched PubMed and PsycInfo for published systematic reviews and the gray literature for unpublished systematic reviews as of June 24, 2020. Reviews were classified as reliable according to several quality criteria, such as whether they conducted a risk of bias assessment of studies and whether they clearly described how outcomes from each study were selected. We assessed possible selective outcome reporting among the reviews. Seven reviews that included a total of 196 studies met inclusion criteria. The overall number of participants was not calculable because not all reviews reported these data. All reviews specified eligibility criteria for studies, but not for outcomes within studies. Only one review assessed risk of bias. Evidence of selective outcome reporting was found in all reviews-only one review incorporated any of the null findings from studies it included. Thus, none were classified as reliable according to prespecified quality criteria. Systematic reviews related to gaming disorder do not meet methodological standards. As clinical and policy decisions are heavily reliant on reliable, accurate, and unbiased evidence synthesis; researchers, clinicians, and policymakers should consider the implications of selective outcome reporting. Limitations of the current summary include using counts of associations and restricting to systematic reviews published in English. Systematic reviewers should follow established guidelines for review conduct and transparent reporting to ensure evidence about technology use disorders is reliable.

Authorship diversity among systematic reviews in eyes and vision

IMPORTANCE

The inclusion of authors from diverse backgrounds and with different lived experiences is critical to ensuring the questions addressed in systematic reviews (SRs), as well as the subsequent conclusions and recommendations made, are representative of the global community.

OBJECTIVE

To assess the gender and geographic diversity of authors among all Cochrane SRs in eyes and vision as compared with a random sample of non-Cochrane SRs of interventions in the field of eyes and vision.

DESIGN

The Cochrane Eyes and Vision US Satellite maintains a database of SRs in the field of eyes and vision. We selected all (n = 313) eyes and vision intervention SRs published in The Cochrane Library and a random sample of 313 eyes and vision intervention SRs published elsewhere for this study. We determined gender of the first and corresponding authors ("woman," "man," or "unknown") using a previously developed algorithm and their location based on institution country and the World Health Organization region.

RESULTS

From the 626 reviews included in our sample, we identified 751 unique authors who comprised 887 author positions (i.e., first and/or corresponding authors). We were able to ascertain the gender of 647/751 (86%) authors: 276 women and 371 men. Among Cochrane eyes and vision SRs, the proportions of women in first and/or corresponding author positions were consistent and approximately equal to men. Among non-Cochrane eyes and vision SRs, the representation of women was markedly lower as corresponding authors than other positions. Most authors of Cochrane eyes and vision SRs were from the UK (31%) and USA (26%), whereas most authors of non-Cochrane SRs were from China (34%).

CONCLUSIONS AND RELEVANCE

Compared with authors of non-Cochrane SRs in eyes and vision, authors of Cochrane SRs appear to have approximately equal representation of women and men among perceived important author positions and be located in European and North American countries, possibly due to the locations of the Cochrane editorial teams. Cochrane Eyes and Vision should continue to recruit authors from around the world in locations that reflect the global burden of eye disease.

Strengthening the integration of eye care into the health system: methodology for the development of the WHO package of eye care interventions

Objective

To describe the rational for, and the methods that will be employed to develop, the WHO package of eye care interventions (PECI).

Methods and analysis

The development of the package will be conducted in four steps: (1) selection of eye conditions (for which interventions will be included in the package) based on epidemiological data on the causes of vision impairment and blindness, prevalence estimates of eye conditions and health facility data; (2) identification of interventions and related evidence for the selected eye conditions from clinical practice guidelines and high-quality systematic reviews by a technical working group; (3) expert agreement on the inclusion of eye care interventions in the package and the description of resources required for the provision of the selected interventions; and (4) peer review. The project will be led by the WHO Vision Programme in collaboration with Cochrane Eyes and Vision. A Technical Advisory Group, comprised of public health and clinical experts in the field, will provide technical input throughout all stages of development.

Results

After considering the feedback of Technical Advisory Group members and reviewing-related evidence, a final list of eye conditions for which interventions will be included in the package has been collated.

Conclusion

The PECI will support Ministries of Health in prioritising, planning, budgeting and integrating eye care interventions into health systems. It is anticipated that the PECI will be available for use in 2021.

Reliability of the Evidence Addressing Treatment of Corneal Diseases: A Summary of Systematic Reviews

Importance

Patient care should be informed by clinical practice guidelines, which in turn should be informed by evidence from reliable systematic reviews. The American Academy of Ophthalmology is updating its Preferred Practice Patterns (PPPs) for the management of the following 6 corneal diseases: bacterial keratitis, blepharitis, conjunctivitis, corneal ectasia, corneal edema and opacification, and dry eye syndrome.

Objective

To summarize the reliability of the existing systematic reviews addressing interventions for corneal diseases.

Data Source

The Cochrane Eyes and Vision US Satellite database.

Study Selection

In this study of published systematic reviews from 1997 to 2017 (median, 2014), the Cochrane Eyes and Vision US Satellite database was searched for systematic reviews evaluating interventions for the management of any corneal disease, combining eyes and vision keywords and controlled vocabulary terms with a validated search filter.

Data Extraction and Synthesis

The study classified systematic reviews as reliable when each of the following 5 criteria were met: the systematic review specified eligibility criteria for inclusion of studies, conducted a comprehensive literature search for studies, assessed risk of bias of the individual included studies, used appropriate methods for quantitative syntheses (meta-analysis) (only assessed if meta-analysis was performed), and had conclusions that were supported by the results of the systematic review. They were classified as unreliable if at least 1 criterion was not met.

Main Outcomes and Measures

The proportion of systematic reviews that were reliable and the reasons for unreliability.

Results

This study identified 98 systematic reviews that addressed interventions for 15 corneal diseases. Thirty-three of 98 systematic reviews (34%) were classified as unreliable. The most frequent reasons for unreliability were that the systematic review did not conduct a comprehensive literature search for studies (22 of 33 [67%]), did not assess risk of bias of the individual included studies (13 of 33 [39%]), and did not use appropriate methods for quantitative syntheses (meta-analysis) (12 of 17 systematic reviews that conducted a quantitative synthesis [71%]). Sixty-five of 98 systematic reviews (66%) were classified as reliable. Forty-two of the 65 reliable systematic reviews (65%) addressed corneal diseases relevant to the 2018 American Academy of Ophthalmology PPPs; 33 of these 42 systematic reviews (79%) are cited in the 2018 PPPs.

Conclusions and Relevance

One in 3 systematic reviews addressing interventions for corneal diseases are unreliable and thus were not used to inform PPP recommendations. Careful adherence by systematic reviewers and journal editors to well-established best practices regarding systematic review conduct and reporting might help make future systematic reviews in eyes and vision more reliable.

Systematic reviews and randomised controlled trials on open angle glaucoma

Evidence for the effectiveness of interventions to prevent the progression of optic nerve damage in open angle glaucoma has evolved over the last 25 years. We describe that evolution through the systematic reviews on various aspects of the topic and how those reviews have highlighted the need for new trials. Though we can be confident that lowering pressure does indeed reduce the risk of progression, we still lack good evidence on the comparative effectiveness of different treatments not so much on lowering pressure but on preventing progression of the disease. This is true for different medicines, types of laser and especially for different surgical interventions. As always there is a need for more research, but this needs to be focussed on key uncertainties using core outcome sets which avoid research waste. Ultimately, our guidelines can be based on sound and comprehensive evidence of effectiveness.

Cochrane Eyes and Vision: a perspective introducing Cochrane Corner in Eye

In 1972, Archie Cochrane wrote "It is surely a great criticism of our profession that we have not organised a critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomised controlled trials". The Cochrane Collaboration arose in response to Archie Cochrane's challenge. Cochrane Eyes and Vision aims to prepare and promote access to systematic reviews of interventions for preventing or treating eye conditions and/or visual impairment, and helping people adjust to visual impairment or blindness. To identify all relevant randomised controlled trials, Cochrane Eyes and Vision has a team of information specialists who develop search strategies to identify studies for inclusion in Cochrane reviews. Since 1997 we have published 266 protocols, 193 new reviews and 158 updated reviews. The majority of these are reviews of intervention effectiveness; three reviews are diagnostic test accuracy reviews. Overall 18% of reviews contain no trials, highlighting a potential evidence gap. We provide training, education and guidance to systematic review authors and work with clinical and patient partners to prioritise and disseminate reviews. In addition, Cochrane Eyes and Vision US satellite carries out critical methodologic research addressing topics relevant to producing high-quality reviews. We are partnering with the journal Eye to publish commentaries on selected Cochrane systematic review findings. This partnership will allow us to make high-quality evidence available to ophthalmologists and other practitioners, researchers, policy makers and patients.