A cross-sectional analysis of reporting guideline and clinical trial registration policies in nephrology journals

OBJECTIVE

To determine the extent to which nephrology journals recommend and require reporting guideline adherence and clinical trial registration.

BACKGROUND

Despite a rising disease burden, research published on chronic kidney disease (CKD) and the field of nephrology has failed to keep pace and is limited. To improve the quality of research in the field of nephrology, reporting guidelines have been developed to minimize such deficits in research quality. However, the extent to which nephrology journals require and use reporting guidelines in addition to clinical trial registration is unknown.

METHODS

Sixty-two Nephrology journals were selected through the 2021 Scopus CiteScore tool. Each journal's Instructions for Authors was assessed to determine endorsement of study design-specific reporting guidelines or clinical trial registration. Researchers used R (version 4.2.1) and RStudio to create data summaries of descriptive statistics for nephrology journal reporting guidelines.

RESULTS

Clinical trial registration was required by 52% (32/62) of nephrology journals within our sample. The reporting guideline for clinical trials, CONSORT, was required by 17.74% (11/62) of journals. The EQUATOR Network was mentioned by 46.77% (29/62) of journals, while 9.67% (6/62) failed to mention the ICMJE. The reporting guideline for systematic review, PRISMA, was only required by 12.90% (8/62) of journals. When contacting journal editors, 9.67% (6/62) responded and 4.83% (3/62) provided clarifying information.

CONCLUSIONS

Reporting guidelines and clinical trial registration are suboptimally required and recommended by nephrology journals. Their adoption may decrease bias and increase research quality. Thus, nephrology journals should consider a more complete endorsement of these safeguards.

Determining the Completeness of Registration and Reporting in Systematic Reviews of Yoga for Health

Introduction: Yoga is a mind-body practice often used to improve health. Systematic reviews (SRs) of randomized controlled trials on yoga for health are foundational to evidence-based yoga interventions and require rigorous and transparent methods, including preparation of a protocol (e.g., PROSPERO) and following SR reporting guidelines (e.g., Preferred Reporting Items for Systematic Reviews and Meta-Analyses [PRISMA]). Objective: To evaluate the availability of protocols and the completeness of reporting for SRs on yoga for health. Methods: We used a previously assembled database of SRs focused on the use of yoga for health. The authors independently extracted data on protocol availability, PROSPERO registration, and reporting for each PRISMA 2009 checklist item. Discrepancies were discussed or referred to a third author. We used Stata 10 software to produce descriptive statistics and tests for relationships between registration, reporting, and publication year, country, and journal type. Results: We included 147 reviews published between 2005 and 2019. The most common first author country was the United States or Germany (total 67/147; 46%), and the most common journal type was specialty journals (71/147; 48%). Most reviews (116/147; 79%) made no mention of a protocol or registration, and only 15/147 (10%) reviews were linked to an accessible protocol or registration. Most SRs published in 2010 or later mentioned or cited PRISMA (97/139; 70%), and individual PRISMA items were addressed between 10% and 100% of the time. PRISMA reporting improved; over time, but there was no relationship with country or journal type. Discussion: This study identifies a need for increased SR registration for yoga research. The assessment of PRISMA reporting did not evaluate the comprehensiveness with which each item was reported, and while trends are encouraging, there is likely room for improvement. We recommend registering all yoga SRs and following updated PRISMA and recent yoga-specific guidelines for reporting. This may increase transparency, minimize bias, and produce high-quality data to inform evidence-based yoga practices.

Effect of PRISMA 2009 on reporting quality in systematic reviews and meta-analyses in high-impact dental medicine journals between 1993-2018

INTRODUCTION

The PRISMA guidelines were published in 2009 to address inadequate reporting of key methodological details in systematic reviews and meta-analyses (SRs/MAs). This study sought to assess the impact of PRISMA on the quality of reporting in the full text of dental medicine journals.

METHODS

This study assessed the impact of PRISMA (2009) on thirteen methodological details in SRs/MAs published in the highest-impact dental medicine journals between 1993-2009 (n = 211) and 2012-2018 (n = 618). The study further examined the rate of described use of PRISMA in the abstract or full text of included studies published post- PRISMA and the impact of described use of PRISMA on level of reporting. This study also examined potential effects of inclusion of PRISMA in Instructions for Authors, along with study team characteristics.

RESULTS

The number of items reported in SRs/MAs increased following the publication of PRISMA (pre-PRISMA: M = 7.83, SD = 3.267; post-PRISMA: M = 10.55, SD = 1.4). Post-PRISMA, authors rarely mention PRISMA in abstracts (8.9%) and describe the use of PRISMA in the full text in 59.87% of SRs/MAs. The described use of PRISMA within the full text indicates that its intent (guidance for reporting) is not well understood, with over a third of SRs/MAs (35.6%) describing PRISMA as guiding the conduct of the review. However, any described use of PRISMA was associated with improved reporting. Among author team characteristics examined, only author team size had a positive relationship with improved reporting.

CONCLUSION

Following the 2009 publication of PRISMA, the level of reporting of key methodological details improved for systematic reviews/meta-analyses published in the highest-impact dental medicine journals. The positive relationship between reference to PRISMA in the full text and level of reporting provides further evidence of the impact of PRISMA on improving transparent reporting in dental medicine SRs/MAs.

Completeness of reporting in systematic reviews and meta-analyses in vascular surgery

OBJECTIVE

Systematic reviews (SRs) and meta-analyses are essential in informing clinical guidelines and decision-making. Complete reporting of SRs through compliance to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines promotes transparency, reproducibility, and consistency across the literature. The purpose of this meta-epidemiological study is to assess the completeness of reporting of SRs in the vascular surgery literature.

METHODS

MEDLINE and Embase were used to search through four major vascular surgery journals and four high impact general medical journals for SRs published between 2018 and October 2022 evaluating clinical treatments for any pathology treated by a vascular surgeon. Data screening and extraction were conducted in duplicate. The reporting completeness of each included SR was measured with reference to the 27-item PRISMA checklist, and methodological quality was evaluated using the Assessing the Methodological Quality of Systematic Reviews 2 (AMSTAR 2) tool. Parametric tests were used to evaluate for associations between PRISMA score and study funding, protocol registration, publication in a higher impact factor journal, and AMSTAR 2 score. The protocol is available online: 10.17605/OSF.IO/VBC5N.

RESULTS

Of 1653 articles captured in the initial search, 162 SRs were included in the final analysis. All SRs had more than one incomplete PRISMA item. The mean PRISMA score was 21.2/27 (standard deviation: 2.9, 78.5% compliance), and the mean AMSTAR 2 score was 11.7/16 (standard deviation: 1.9, 73%). SRs that had a prospectively registered protocol had a higher PRISMA score (22.9 vs 20.6, P < .001) as did those that were published in higher impact factor journals (23.3 vs 21.0, P = .017). There was a large positive correlation between an SR's PRISMA and AMSTAR 2 scores (Pearson r = 0.655, 95% confidence interval: 0.55-0.74). There were no associations between the PRISMA score and publication year (P = .067) or funding status (P = .076).

CONCLUSIONS

Overall, the reporting of SRs and meta-analyses in vascular surgery is less than ideal, with several key items being consistently under-reported. Prospective registration and methodological quality as measured by AMSTAR 2 scores are positively associated with improved reporting. Authors, reviewers, and journal editors should consider these findings moving forward to encourage completeness of SR reporting. Raising awareness surrounding the value of complete reporting of SRs can aid in enhancing the quality of evidence, and journals should consider these findings in methods used to promote SR reporting.

A meta-evaluation of the quality of reporting and execution in ecological meta-analyses

Quantitatively summarizing results from a collection of primary studies with meta-analysis can help answer ecological questions and identify knowledge gaps. The accuracy of the answers depends on the quality of the meta-analysis. We reviewed the literature assessing the quality of ecological meta-analyses to evaluate current practices and highlight areas that need improvement. From each of the 18 review papers that evaluated the quality of meta-analyses, we calculated the percentage of meta-analyses that met criteria related to specific steps taken in the meta-analysis process (i.e., execution) and the clarity with which those steps were articulated (i.e., reporting). We also re-evaluated all the meta-analyses available from Pappalardo et al. [1] to extract new information on ten additional criteria and to assess how the meta-analyses recognized and addressed non-independence. In general, we observed better performance for criteria related to reporting than for criteria related to execution; however, there was a wide variation among criteria and meta-analyses. Meta-analyses had low compliance with regard to correcting for phylogenetic non-independence, exploring temporal trends in effect sizes, and conducting a multifactorial analysis of moderators (i.e., explanatory variables). In addition, although most meta-analyses included multiple effect sizes per study, only 66% acknowledged some type of non-independence. The types of non-independence reported were most often related to the design of the original experiment (e.g., the use of a shared control) than to other sources (e.g., phylogeny). We suggest that providing specific training and encouraging authors to follow the PRISMA EcoEvo checklist recently developed by O'Dea et al. [2] can improve the quality of ecological meta-analyses.

Do Meta-Analyses of Intervention/Prevention Programs in the Field of Criminology Meet the Tests of Transparency and Reproducibility?

While assessments of transparent reporting practices in meta-analyses are not uncommon in the field of health sciences interventions, they are limited in the social sciences and to our knowledge are non-existent in criminology. Modified PRISMA 2020 checklists were used to assess transparency and reproducibility of reporting for a sample of 33 meta-analyses of intervention/prevention evaluations published in scholarly journals between 2016 and 2021. Results indicate that the average rate of transparent reporting practices was 63%; adherence varied considerably across studies and subscales, with low rates of adherence for some core checklist items. Overwhelmingly, studies were not reproducible in their entirety; article word count was significantly correlated with reproducibility (r = 0.4028, p < .03). These findings suggest that substantial changes to reporting practices are necessary to meet traditional meta-analytic claims of transparency and reproducibility. Study limitations include sample size, coding instruments, and coding subjectivity.

Completeness of reporting in abstracts of randomized controlled trials assessing interventional radiology for liver disease

PURPOSE

To assess the completeness of reporting in abstracts of published randomized controlled trials (RCTs) assessing interventional radiology (IR) for liver disease; to assess whether publication of the 2017 CONSORT update for nonpharmacological treatments (NPT) resulted in changes in abstract reporting; and to identify factors associated with better reporting.

MATERIAL AND METHODS

MEDLINE and Embase were searched to identify RCTs of IR for liver disease (January 2015-September 2020). Two reviewers assessed the completeness of abstract reporting according to the CONSORT-NPT-2017-update. The primary outcome was the mean number of CONSORT items completely reported among 10 items reported in <50% of the abstracts published in 2015. A time series analysis assessed the evolution trend over time. A multivariate regression model was used to identify factors associated with better reporting.

RESULTS

A total of 107 abstracts of RCTs published in 61 journals were included. Overall, 74% (45/61) of journals endorsed the main CONSORT guidelines, of which 60% (27/45) had a policy to implement them. The mean number of primary outcome items completely reported increased by 0.19 over the study period. The publication of the CONSORT-NPT update did not lead to an increase in the trend of items reported (increase of 0.04 items/month before vs. 0.02 after, P=0.41). Factors associated with more complete reporting were impact factor (OR=1.13; 95%CI:1.07-1.18) and endorsement of CONSORT with an implementation policy (OR=8.29; 95%CI:2.04-33.65).

CONCLUSION

Completeness of reporting is incomplete in abstracts of trials of IR liver disease and did not improve after publication of the CONSORT-NPT-2017 update with abstract guidance.

How to Critically Appraise and Interpret Systematic Reviews and Meta-Analyses of Diagnostic Accuracy: A User Guide

Systematic reviews of diagnostic accuracy studies can provide the best available evidence to inform decisions regarding the use of a diagnostic test. In this guide, the authors provide a practical approach for clinicians to appraise diagnostic accuracy systematic reviews and apply their results to patient care. The first step is to identify an appropriate systematic review with a research question matching the clinical scenario. The user should evaluate the rigor of the review methods to evaluate its credibility (Did the review use clearly defined eligibility criteria, a comprehensive search strategy, structured data collection, risk of bias and applicability appraisal, and appropriate meta-analysis methods?). If the review is credible, the next step is to decide whether the diagnostic performance is adequate for clinical use (Do sensitivity and specificity estimates exceed the threshold that makes them useful in clinical practice? Are these estimates sufficiently precise? Is variability in the estimates of diagnostic accuracy across studies explained?). Diagnostic accuracy systematic reviews that are judged to be credible and provide diagnostic accuracy estimates with sufficient certainty and relevance are the most useful to inform patient care. This review discusses comparative, noncomparative, and emerging approaches to systematic reviews of diagnostic accuracy using a clinical scenario and examples based on recent publications.

Transparency and completeness of reporting of depression screening tool accuracy studies: A meta-research review of adherence to the Standards for Reporting of Diagnostic Accuracy Studies statement

OBJECTIVES

Accurate and complete study reporting allows evidence users to critically appraise studies, evaluate possible bias, and assess generalizability and applicability. We evaluated the extent to which recent studies on depression screening accuracy were reported consistent with Standards for Reporting of Diagnostic Accuracy Studies (STARD) statement requirements.

METHODS

MEDLINE was searched from January 1, 2018 through May 21, 2021 for depression screening accuracy studies.

RESULTS

106 studies were included. Of 34 STARD items or sub-items, the number of adequately reported items per study ranged from 7 to 18 (mean = 11.5, standard deviation [SD] = 2.5; median = 11.5), and the number inadequately reported ranged from 3 to 17 (mean = 10.1, SD = 2.5; median = 10.0). There were eight items adequately reported, seven partially reported, 11 inadequately reported, and four not applicable in ≥50% of studies; the remaining four items had mixed reporting. Items inadequately reported in ≥70% of studies related to the rationale for index test cut-offs examined, missing data management, analyses of variability in accuracy results, sample size determination, participant flow, study registration, and study protocol.

CONCLUSION

Recently published depression screening accuracy studies are not optimally reported. Journals should endorse and implement STARD adherence.

A systematic evaluation of methodological and reporting quality of meta-analysis published in the field of gastrointestinal endoscopy

BACKGROUND

To evaluate the methodological and reporting quality of published meta-analyses (MAs) in four major gastrointestinal endoscopic journals, and identify the predicted factors for high quality.

METHODS

A systematic search was performed in PubMed to identify MAs from 1, January, 2016 to 31, December, 2020 in four major gastrointestinal endoscopic journals (including Digestive Endoscopy, Gastrointestinal Endoscopy, Surgical Endoscopy, and Endoscopy). We collected the characteristics of MAs after filtering unqualified articles, and assessed methodological and reporting qualities for eligible articles by AMSTAR tool and PRISMA checklist, respectively. Logistic regression was used for identifying predictive factors for high quality.

RESULTS

A total of 289 MAs were identified after screening by predefined inclusion and exclusion criteria. The scores (mean ± SD) of AMSTAR and PRISMA were 7.73 ± 1.11 and 22.90 ± 1.85, respectively. In PRISMA checklist, some items had less than 50% complete adherence, including item 2 (structured summary), items 5 (protocol and registration), items 12 and 19 (risk of bias in studies), item 27 (funding support). Item 1 (a priori design), item 4 (gray literature research), item 5 (list of included and excluded) were inferior to 50% adherence in AMSTAR tool. We found the predictive factors for high quality through logistic regression analysis: a priori design and funding support were associated with methodological quality. Protocol and registration influenced the methodological and reporting quality closely.

CONCLUSION

In general, qualities on the methodology and the reporting of MAs published in the gastrointestinal endoscopic journals are good, but both of which still potentially need further improvement.

Measuring quality of reporting in systematic reviews of diagnostic test accuracy studies in medical imaging: comparison of PRISMA-DTA and PRISMA

OBJECTIVES

To compare the reporting quality measured by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses of Diagnostic Test Accuracy studies (PRISMA-DTA) vs the original PRISMA checklist for systematic reviews of diagnostic test accuracy studies in imaging and survey the use of PRISMA-DTA by researchers and endorsement by journals.

METHODS

Systematic reviews of DTA studies published in 2020 and 2021 in Quartile 1 and Quartile 3 medical imaging journals (defined by Journal Citation Reports) were identified through PubMed. The reporting of each systematic review was assessed using PRISMA-DTA, PRISMA-2009 and PRISMA-2020. The item scores and overall score were compared among the three checklists. We also examined checklist adoption by the included systematic reviews and surveyed checklist endorsement from author instructions of included journals.

RESULTS

A total of 173 systematic reviews from 66 journals were included. The use of PRISMA-DTA, compared with PRISMA-2009 and PRISMA-2020, identified more issues in the reporting of title (proportion of systematic reviews with proper reporting, 27.2% vs 98.8% vs 98.8%), abstract (39.3% vs 97.1% vs 64.7%), eligibility criteria (67.6% vs 94.2% vs 94.2%), search (28.9% vs 72.3% vs 28.9%), definitions for data extraction (14.5% vs 91.9% vs 91.9%), diagnostic accuracy measures (38.2% vs 93.6% vs 93.6%), synthesis of results (28.9% vs 89.6% vs 73.4%) and results of individual studies (40.5% vs 80.3% vs 80.3%). The overall median reporting score measured by PRISMA-DTA (72.0% (interquartile range (IQR), 66.7-77.8%)) was lower than that measured by PRISMA-2009 (88.9% (IQR, 84.0-92.6%)) and similar to that measured by PRISMA-2020 (74.1% (IQR, 66.7-77.8%)). Additionally, PRISMA-DTA was used by only 43 (24.9%) systematic reviews and endorsed by two (3.0%) journals. These trends remained consistent for reviews published in journals with diverse scientific impact.

CONCLUSIONS

The use of PRISMA-DTA may identify more reporting inadequacies compared with the original PRISMA checklists when assessing diagnostic test accuracy systematic reviews, especially in critical sections such as title, abstract and methods. However, this tool is not commonly used by researchers and is inadequately endorsed by imaging journals. Our findings suggest a strong need to use PRISMA-DTA for reporting of diagnostic test accuracy systematic reviews by authors and its endorsement by journals. © 2022 International Society of Ultrasound in Obstetrics and Gynecology.

Completeness of Reporting of Systematic Reviews and Meta-Analysis of Diagnostic Test Accuracy (DTA) of Radiological Articles Based on the PRISMA-DTA Reporting Guideline

RATIONALE AND OBJECTIVES

This study evaluated the completeness of systematic reviews and meta-analyses in radiology using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses for Diagnostic Test Accuracy (PRISMA-DTA) and PRISMA-DTA for Abstracts guidelines between articles published before and those published after the issuance of the guideline and identify areas that have been poorly reported.

MATERIALS AND METHODS

PubMed were searched for systematic reviews on DTA with or without meta-analyses published in general radiology journals between January 1, 2016 and December 31, 2020. The identified articles were assessed for completeness of reporting according to the PRISMA-DTA. Subgroup analyses were performed for association of completeness of reporting with multiple cofactors.

RESULTS

The search identified 183 reviews from 12 journals. The mean numbers (standard deviation) of reported PRISMA-DTA and PRISMA-DTA for Abstracts items in the full texts and abstracts were 18.45 (2.02) and 5.66 (1.28), respectively. Subgroup analysis showed that compared to the corresponding reference groups, a higher mean number of reported PRISMA-DTA items was associated with publication during July 2018-December 2020 [(17.82 (2.01) vs 18.89 (1.91); p = 0.034), citation of the PRISMA-DTA [17.62 (1.86) vs 20.27 (2.02); p < 0.001], and inclusion of supplementary materials [17.64 (2) vs 19.09 (1.8); p < 0.001] on multiple-linear regression analysis.

CONCLUSION

Completeness of reporting with respect to the PRISMA-DTA and PRISMA-DTA for Abstracts has improved modestly since the publication of the PRISMA-DTA guideline; however, increasing awareness of the specific weakness provides the chance for completeness improvement.

Prevalence of brucellosis in livestock of African and Asian continents: A systematic review and meta-analysis

Brucellosis is a highly contagious bacterial disease that mainly affects ruminants, but it may affect equines, canines, and felines. The disease is of utmost significance from an economic standpoint in countries where there is no national brucellosis prevention and eradication policy in operation. A systematic review was done to estimate disease burden, incidences, prevalence, and geographical distribution critical in planning appropriate intervention strategies for the control and prevention of Brucellosis. Research articles that were published during the period 2000–2020 were considered for this study after reinforced scrutiny by two independent authors. Meta-regression was used to examine heterogeneity, and subgroup and sensitivity analyses were used to calculate residual heterogeneity and the pooled prevalence of Brucellosis in livestock. Confounders such as geography, a diagnostic test, and species had the greatest R2 values of 17.8, 8.8, and 2.3%, respectively, indicating the presence of heterogeneity and necessitating more research into sensitivity and subgroup analysis. The combined pooled prevalence of brucellosis in both Asia and African countries was 8% when compared to 12% in the Indian livestock population. The findings of our systematic review and meta-analysis indicate that brucellosis continues to be an important animal and public health concern in developing countries of Asia and Africa, as evidenced by the prevalence rate of brucellosis in these regions. Our findings suggested that well-planned epidemiological surveillance studies in different geographic settings are needed to generate reliable data on disease burden including the economic loss in Asian and African countries.

Adherence to the PRISMA statement and its association with risk of bias in systematic reviews published in rehabilitation journals: A meta-research study

BACKGROUND

Systematic reviews (SRs) and meta-analyses are essential resources for the clinicians. They allow to evaluate the strengths and the weaknesses of the evidence to support clinical decision-making if they are adequately reported. Little is known in the rehabilitation field about the completeness of reporting of SRs and its relationship with the risk of bias (ROB).

OBJECTIVES

Primary: 1) To evaluate the completeness of reporting of systematic reviews (SRs) published in rehabilitation journals by evaluating their adherence to the PRISMA 2009 checklist, 2) To investigate the relationship between ROB and completeness of reporting. Secondary: To study the association between completeness of reporting and journals and study characteristics.

METHODS

A random sample of 200 SRs published between 2011 and 2020 in 68 rehabilitation journals was indexed under the "rehabilitation" category in the InCites database. Two independent reviewers evaluated adherence to the PRISMA checklist and assessed ROB using the ROBIS tool. Overall adherence and adherence to each PRISMA item and section were calculated. Regression analyses investigated the association between completeness of reporting, ROB, and other characteristics (impact factor, publication options, publication year, and study protocol registration).

RESULTS

The mean overall PRISMA adherence across the 200 studies considered was 61.4%. Regression analyses show that having a high overall ROB is a significant predictor of lower adherence (B=-7.1%; 95%CI -12.1, -2.0). Studies published in fourth quartile journals displayed a lower overall adherence (B= -7.2%; 95%CI -13.2, -1.3) than those published in first quartile journals; the overall adherence increased (B= 11.9%; 95%CI 5.9, 18.0) if the SR protocol was registered. No association between adherence, publication options, and publication year was found.

CONCLUSION

Reporting completeness in rehabilitation SRs is suboptimal and is associated with ROB, impact factor, and study registration. Authors of SRs should improve adherence to the PRISMA guideline, and journal editors should implement strategies to optimize the completeness of reporting.

Quality Reporting of Systematic Review and Meta-Analysis According to PRISMA 2020 Guidelines: Results from Recently Published Papers in the Korean Journal of Radiology

OBJECTIVE

To evaluate the completeness of the reporting of systematic reviews and meta-analyses published in a general radiology journal using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 guidelines.

MATERIALS AND METHODS

Twenty-four articles (systematic review and meta-analysis, n = 18; systematic review only, n = 6) published between August 2009 and September 2021 in the Korean Journal of Radiology were analyzed. Completeness of the reporting of main texts and abstracts were evaluated using the PRISMA 2020 statement. For each item in the statement, the proportion of studies that met the guidelines' recommendation was calculated and items that were satisfied by fewer than 80% of the studies were identified. The review process was conducted by two independent reviewers.

RESULTS

Of the 42 items (including sub-items) in the PRISMA 2020 statement for main text, 24 were satisfied by fewer than 80% of the included articles. The 24 items were grouped into eight domains: 1) assessment of the eligibility of potential articles, 2) assessment of the risk of bias, 3) synthesis of results, 4) additional analysis of study heterogeneity, 5) assessment of non-reporting bias, 6) assessment of the certainty of evidence, 7) provision of limitations of the study, and 8) additional information, such as protocol registration. Of the 12 items in the abstract checklists, eight were incorporated in fewer than 80% of the included publications.

CONCLUSION

Several items included in the PRISMA 2020 checklist were overlooked in systematic review and meta-analysis articles published in the Korean Journal of Radiology. Based on these results, we suggest a double-check list for improving the quality of systematic reviews and meta-analyses. Authors and reviewers should familiarize themselves with the PRISMA 2020 statement and check whether the recommended items are fully satisfied prior to publication.

Reporting completeness in abstracts of systematic reviews of diagnostic test accuracy studies in cardiovascular diseases is suboptimal

OBJECTIVE

Journal abstracts are crucial for the identification and initial assessment of content of studies. We evaluated whether authors in the field of cardiovascular diseases (CVDs) reported Diagnostic Test Accuracy Systematic Reviews (DTA SRs) abstracts adequately, as defined by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)-DTA guidelines.

METHODS

SRs of DTA studies in CVDs published in general and specialized medical journals were identified in a MEDLINE search between 2010-2020. Adherence to 12 PRISMA-DTA for abstracts items was assessed independently by two reviewers and compared by journal's type. Moreover, the association of reporting completeness with different characteristics was investigated.

RESULTS

We included 72 abstracts. Studies published in general medical journals had higher mean reporting score than those in specialized journals (6.2 vs 5.3 out of 12 items; mean difference: 0.88; 95% confidence interval: 0.21, 1.55). PRISMA-DTA adherence was higher in journals that adopted this guideline and in articles with structured abstracts. However, number of participants analysed, funding and registration were the least-reported items in the identified abstracts.

CONCLUSION

The reporting of abstracts of DTA reports in CVDs is suboptimal according to PRISMA-DTA guidelines. Abstract reporting could be improved with the use of higher word count limits and the adoption of PRISMA-DTA guidelines especially in specialized journals.

Predictors of Higher Quality of Systematic Reviews Addressing Nutrition and Cancer Prevention

Systematic reviews/meta-analyses (SR/MAs) are considered a reliable source of information in healthcare. We aimed to explore the association of several characteristics of SR/MAs addressing nutrition in cancer prevention and their quality/risk of bias (using assessments from AMSTAR-2 and ROBIS tools). The analysis included 101 SR/MAs identified in a systematic survey. Associations of each specified characteristic (e.g., information about the protocol, publication year, reported use of GRADE, or other methods for assessing overall certainty of evidence) with the number of AMSTAR-2 not met (‘No’ responses) and the number of ROBIS items met (‘Probably Yes’ or “Yes’ responses) were examined. Poisson regression was used to identify predictors of the number of ‘No’ answers (indicating lower quality) for all AMSTAR-2 items and the number of ‘Yes’ or ‘Probably Yes’ answers (indicating higher quality/lower concern for bias) for all ROBIS items. Logistic regression was used to identify variables associated with at least one domain assessed as ‘low concern for bias’ in the ROBIS tool. In multivariable analysis, SR/MAs not reporting use of any quality/risk of bias assessment instrument for primary studies were associated with a higher number of ‘No’ answers for all AMSTAR-2 items (incidence rate ratio (IRR) 1.26, 95% confidence interval (CI) 1.09–1.45), and a lower number of ‘Yes’ or ‘Probably Yes’ answers for all ROBIS items (IRR 0.76, 95% CI 0.66–0.87). Providing information about the protocol and search for unpublished studies was associated with a lower number of ‘No’ answers (IRR 0.73, 95% CI 0.56–0.97 and IRR 0.75, 95% CI 0.59–0.95, respectively) and a higher number of ‘Yes’ or ‘Probably Yes’ answers (IRR 1.43, 95% CI 1.17–1.74 and IRR 1.28, 95% CI 1.07–1.52, respectively). Not using at least one quality/risk of bias assessment tool for primary studies within an SR/MA was associated with lower odds that a study would be assessed as ‘low concern for bias’ in at least one ROBIS domain (odds ratio 0.061, 95% CI 0.007–0.527). Adherence to methodological standards in the development of SR/MAs was associated with a higher overall quality of SR/MAs addressing nutrition for cancer prevention.

High variability in results and methodological quality among overlapping systematic reviews on the same topics in surgery: a meta-epidemiological study

BACKGROUND

Redundant publication of systematic reviews and meta-analyses (SRs/MAs) on the same topic presents an increasing burden for clinicians. The aim of this study was to describe variabilities in effect size and methodological quality of overlapping surgery-related SRs/MAs and to investigate factors associated with their postpublication citations.

METHODS

PubMed/MEDLINE was searched to identify SRs/MAs of RCTs on thoracoabdominal surgeries published in 2015. Previous SRs/MAs on the same topics published within the preceding 5 years (2011-2015) were identified and 5-year citation counts (through to 2020) were evaluated. Discrepancies in pooled effect sizes and their methodological quality using A Measurement Tool to Assess Systematic Reviews (AMSTAR) among overlapping SRs/MAs were assessed. The SR/MA-level factors associated with 5-year citation counts were explored, using a mixed-effects regression model with a random intercept for surgical topics.

RESULTS

A total of 57 surgery-related SRs/MAs (48 topics) published in 2015 were identified, and 146 SRs/MAs had overlapping publications on 29 topics (60.4 per cent of all topics) in the preceding 5 years. There was considerable variability in methodological quality of SRs/MAs and coverage probability for relevant RCTs, resulting in discrepant effect size estimates for the same topic. High quality (AMSTAR score 8-11) was independently associated with higher 5-year citation counts (coefficient = 32.82; 95 per cent c.i. 15.63 to 50.02; P < 0.001).

CONCLUSION

Overlapping SRs/MAs with high variability in results and methodological quality were common in surgery. A high-quality SR/MA score was an independent predictor of more frequent citations. Researchers and journal editors should concentrate their efforts on limiting publications to higher-quality reviews.

Methodological quality of systematic reviews comprising clinical practice guidelines for cardiovascular risk assessment and management for noncardiac surgery

BACKGROUND

Cardiac assessment in noncardiac surgery clinical practice guidelines should be supported by the highest-quality evidence such as that offered by systematic reviews. Currently, the methodological and reporting quality of these studies remains unknown.

METHODS

We used PubMed to search for all clinical practice guidelines related to perioperative cardiovascular patients undergoing noncardiac surgery from 2010 to 2021. The included clinical practice guidelines were analysed for all systematic reviews and meta-analyses. The primary objective of this study was to determine reporting and methodological quality using the PRISMA (Preferred Reporting Instrument for Systematic Reviews and Meta-Analyses) and AMSTAR-2 (A Measurement Tool to Assess Systematic Reviews-2) instruments. Our secondary objective was to compare systematic reviews conducted by the Cochrane Collaboration with non-Cochrane studies.

RESULTS

Three clinical practice guidelines were included in our study. Within these, 78 systematic reviews were included. PRISMA completion ranged from 34.8% to 100.0% with a mean of 76.9%. AMSTAR-2 completion ranged from 15.6% to 96.9% with a mean of 58.0%. Fifty-four systematic reviews underpinned a clinical practice guidelines recommendation, of which 25 were rated 'critically low' by AMSTAR-2 appraisal. Cochrane systematic reviews typically performed better than non-Cochrane studies, but were a minority of the included studies (10/78).

CONCLUSION

We found deficiencies in several key areas regarding the methodological and reporting qualities of systematic reviews included in cardiac assessment in noncardiac surgery clinical practice guidelines. As these clinical practice guidelines are instrumental to clinical decision-making and patient care in cardiac assessment in noncardiac surgery, we advocate for improved reporting quality among systematic reviews cited as supportive evidence for these recommendations.

Methodological Rigor and Temporal Trends of Cardiovascular Medicine Meta-Analyses in Highest-Impact Journals

Background Well-conducted meta-analyses are considered to be at the top of the evidence-based hierarchy pyramid, with an expansion of these publications within the cardiovascular research arena. There are limited data evaluating the trends and quality of such publications. The objective of this study was to evaluate the methodological rigor and temporal trends of cardiovascular medicine-related meta-analyses published in the highest impact journals. Methods and Results Using the Medline database, we retrieved cardiovascular medicine-related systematic reviews and meta-analyses published in The New England Journal of Medicine, The Lancet, Journal of the American Medical Association, The British Medical Journal, Annals of Internal Medicine, Circulation, European Heart Journal, and Journal of American College of Cardiology between January 1, 2012 and December 31, 2018. Among 6406 original investigations published during the study period, meta-analyses represented 422 (6.6%) articles, with an annual decline in the proportion of published meta-analyses (8.7% in 2012 versus 4.6% in 2018, Ptrend=0.002). A substantial number of studies failed to incorporate elements of Preferred Reporting Items for Systematic Reviews and Meta-Analyses or Meta-Analysis of Observational Studies in Epidemiology guidelines (51.9%) and only a minority of studies (10.4%) were registered in PROSPERO (International Prospective Register of Systematic Reviews). Fewer manuscripts failed to incorporate the Preferred Reporting Items for Systematic Reviews and Meta-Analyses or Meta-Analysis of Observational Studies in Epidemiology elements over time (60.2% in 2012 versus 40.0% in 2018, Ptrend<0.001) whereas the number of meta-analyses registered at PROSPERO has increased (2.4% in 2013 versus 17.5% in 2018, Ptrend<0.001). Conclusions The proportion of cardiovascular medicine-related meta-analyses published in the highest impact journals has declined over time. Although there is an increasing trend in compliance with quality-based guidelines, the overall compliance remains low.

Quality appraisal of systematic reviews on methods of labour induction: a systematic review

PURPOSE

Induction of labour has become more common over the last decade, together with an increase in the number of systematic reviews of the subject. However, with multiple systematic reviews it is necessary to evaluate the methodological rigor to ensure the reliability of conclusions and recommendations for clinical practice. Therefore, the aim of this study was to appraise the quality of systematic reviews that examined the efficacy and/or safety of labour induction methods.

METHODS

An electronic search of MEDLINE, Embase, and the Cochrane Library from 2000 to 2020 was conducted. Study selection, data extraction and quality assessment were conducted using A Measurement Tool to Assess Systematic Reviews (AMSTAR) by two independent reviewers, in duplicate.

RESULTS

The search identified 387 publications, of which 48 studies (13%) met the a priori inclusion criteria. No significant relationships were found between study quality and number of citations, journal impact factor, or publication year.

CONCLUSION

Methodological quality for systematic reviews on the induction of labour were ranked as moderate with no significant changes in quality over the past 2 decades. Publication characteristics are not significantly associated with methodological quality, indicating that healthcare professionals should critically appraise studies before applying them to practice.

'Real-world' observational studies in arrhythmia research: data sources, methodology, and interpretation. A position document from European Heart Rhythm Association (EHRA), endorsed by Heart Rhythm Society (HRS), Asia-Pacific HRS (APHRS), and Latin America HRS (LAHRS)

The field of observational studies or "real world studies" is in rapid development with many new techniques introduced and increased understanding of traditional methods. For this reason the current paper provides an overview of current methods with focus on new techniques. Some highlights can be emphasized: We provide an overview of sources of data for observational studies. There is an overview of sources of bias and confounding. Next There is an overview of causal inference techniques that are increasingly used. The most commonly used techniques for statistical modelling are reviewed with focus on the important distinction of risk versus prediction. The final section provides examples of common problems with reporting observational data.

Evaluation of spin in the abstracts of systematic reviews and meta-analyses of studies on opioid use disorder

BACKGROUND

Spin, or the inappropriate formatting of information to emphasize certain outcomes, should not be present in research. This study focuses on identifying and characterizing the presence of spin in systematic review and meta-analysis abstracts that focus on the treatment of opioid use disorder. Methods: Search strategies were developed to identify studies pertaining to the treatment of opioid use disorder. The studies were then screened by two authors. These qualifying studies were then evaluated for the presence of spin within their abstracts by two trained authors. These studies were also evaluated by the AMSTAR-2 standards to evaluate the quality of the qualifying systematic reviews by two trained reviewers. Results: The sample in this study included 113 systematic reviews and meta-analyses. Spin was present in 20 of these studies (20/113, 17.7%). The most common spin form was spin type 3 (6/20, 30%), followed by types 5 and 9 (both 4/20, 20%), type 6 (3/20, 15%), type 7 (2/20, 10%), and type 8 (1/20, 5%). The remaining spin types 1, 2, and 4 were not present in the sample. Of the 113 included studies, the most common intervention type was pharmacologic (93/113, 82%). No significant association was found between the quality of a systematic review and the presence of spin. Conclusions: Findings in this study show positive trends in prevalence of five forms of spin evaluated in abstracts of systematic reviews and meta-analyses looking at treatments for opioid use disorder. However, study quality had no significant association with the presence of spin. Misrepresentation of results, or spin, may alter a clinician's perceptions about treatment efficacies. Therefore, increasing physician awareness of spin may improve clinical decision-making.

Completeness of reporting for systematic reviews of point-of-care ultrasound: a meta-research study

OBJECTIVES

Systematic reviews are often considered among the highest quality of evidence. Completely reported systematic reviews, however, are required so readers can assess for generalisability of the research to practice and risk of bias. The objective of this study was to assess the completeness of reporting for systematic reviews assessing the diagnostic accuracy of point-of-care ultrasound (POCUS) using the Preferred Reporting Items for Systematic Reviews and Meta-analyses for Diagnostic Test Accuracy (PRISMA-DTA) checklist that was published in 2018.

DESIGN AND SETTING

In this meta-research study, MEDLINE, EMBASE and Cochrane Library databases were searched, with no date restriction, on March 1st, 2020 for systematic reviews assessing the diagnostic accuracy of POCUS. Adherence to PRISMA-DTA for the main text and abstract was scored independently and in duplicate using a modified checklist. Prespecified subgroup analyses were performed.

MAIN OUTCOME MEASURES

The primary outcome was the mean PRISMA-DTA checklist adherence for the full-text and abstract.

RESULTS

A total of 71 studies published from 2008 to 2020 met the inclusion criteria. The overall adherence for the full-text was moderate: 19.8 out of 26.0 items (76%) and for the abstract was 7.0 out of 11.0 items (64%). Although many items in the PRISMA-DTA checklist were frequently reported, several were r infrequently reported (<33% of studies), including item 5 (protocol registration), item D2 (minimally acceptable test accuracy) and item 14 (variability in target condition, index test and reference standards). Subgroup analyses showed a higher PRISMA-DTA mean adherence (SD) for high impact journals (20.9 (2.52) vs 18.9 (1.95); p<0.001), studies including supplemental materials (20.6 (2.48) vs 18.9 (2.28); p=0.004), studies citing adherence to PRISMA reporting guidelines (20.4 (1.95) vs 19.0 (3.00); p=0.038) and studies published in journals endorsing PRISMA guidelines (20.2 (2.47) vs 18.6 (2.37); p=0.025). There was variable adherence based on journal of publication (p=0.006), but not for study population (adult vs paediatric vs mixed) (p=0.62), year of publication (p=0.94), body region (p=0.78) or country (p=0.40). There was no variability in abstract adherence based on whether the abstract was structured with subheadings or not (p=0.25). A Spearman's correlation found moderate correlation between higher word counts and abstractand full-text adherence (R=0.45, p<0.001 and R=0.38, p=0.001), respectively.

CONCLUSIONS

Overall, the reporting of POCUS diagnostic accuracy systematic reviews and meta-analyses was moderate. We identified deficits in several key areas including the preregistration of systematic reviews in an online repository, handling of multiple definitions of target conditions, index tests and reference standards and specifying minimally acceptable test accuracy. Prospective registration of reviews and detailed reporting as per PRISMA-DTA during the research process could improve reporting completeness. At an editorial level, word count and supplemental material limitations may impede reporting completeness, whereas endorsement of reporting guidelines on journal websites could improve reporting.

Methodological quality was critically low in 9/10 systematic reviews in advanced cancer patients-A methodological study

OBJECTIVE

To assess the methodological quality and the consideration of heterogeneity in systematic reviews (SRs).

STUDY DESIGN AND SETTING

We conducted a methodological study (CRD42019134904) and searched three databases from January 2010 to July 2019. Interventional SRs with a statistically significant meta-analysis of at least four randomized controlled trials in advanced cancer patients were included. A MeaSurement Tool to Assess Systematic Reviews (AMSTAR) 2 was used to evaluate the SRs' methodological quality. The consideration of heterogeneity was categorized in clinical or/and methodological heterogeneity and not explored.

RESULTS

From 6234 identified references, 261 SRs were included. Most SRs had a critically low quality (230, 88.1%). The majority of them (209, 80.1%) was classified as critically low because of non-registration (222, 85.1%) combined with the non-reporting of excluded full-texts and missing justifications for exclusion (218, 83.5%). Heterogeneity in trial results was not explored at all in 51 (19.5%) SRs whereas clinical heterogeneity was considered in 117 (44.8%), methodological heterogeneity in 13 (5.0%), and both clinical and methodological heterogeneity in 80 (30.7%) SRs.

CONCLUSION

The consideration of these findings in trainings for review authors and peer reviewers could improve the awareness of quality criteria and the quality of future SRs.

TRIAL REGISTRATION

PROSPERO-ID: CRD42019134904.

Barriers to reporting guideline adherence in point-of-care ultrasound research: a cross-sectional survey of authors and journal editors

OBJECTIVE

Although the literature supporting the use of point-of-care ultrasound (POCUS) continues to grow, incomplete reporting of primary diagnostic accuracy studies has previously been identified as a barrier to translating research into practice and to performing unbiased systematic reviews. This study assesses POCUS investigator and journal editor attitudes towards barriers to adhering to the Standards for Reporting of Diagnostic Accuracy Studies (STARD) 2015 guidelines.

DESIGN, SETTING, PARTICIPANTS

Two separate surveys using a 5-point Likert scale were sent to POCUS study investigators and journal editors to assess for knowledge, attitude and behavioural barriers to the complete reporting of POCUS research. Respondents were identified based on a previous study assessing STARD 2015 adherence for POCUS studies published in emergency medicine, anaesthesia and critical care journals. Responses were anonymously linked to STARD 2015 adherence data from the previous study. Written responses were thematically grouped into the following categories: knowledge, attitude and behavioural barriers to quality reporting, or other. Likert response items are reported as median with IQRs.

MAIN OUTCOME MEASURES

The primary outcome was the median Likert score for the investigator and editor surveys assessing knowledge, attitude and behavioural beliefs about barriers to adhering to the STARD 2015 guidelines.

RESULTS

The investigator survey response rate was 18/69 (26%) and the editor response rate was 5/21 (24%). Most investigator respondents were emergency medicine practitioners (13/21, 62%). Two-thirds of investigators were aware of the STARD 2015 guidelines (12/18, 67%) and overall agreed that incomplete reporting limits generalisability and the ability to detect risk of bias (median 4 (4, 5)). Investigators felt that the STARD 2015 guidelines were useful, easy to find and easy to use (median 4 (4, 4.25); median 4 (4, 4.25) and median 4 (3, 4), respectively). There was a shared opinion held by investigators and editors that the peer review process be primarily responsible for ensuring complete research reporting (median 4 (3, 4) and median 4 (3.75, 4), respectively). Three of 18 authors (17%) felt that the English publication language of STARD 2015 was a barrier to adherence.

CONCLUSIONS

Although investigators and editors recognise the importance of completely reported research, reporting quality is still a core issue for POCUS research. The shared opinion held by investigators and editors that the peer review process be primarily responsible for reporting quality is potentially problematic; we view completely reported research as an integral part of the research process that investigators are responsible for, with the peer review process serving as another additional layer of quality control. Endorsement of reporting guidelines by journals, auditing reporting guideline adherence during the peer review process and translation of STARD 2015 guidelines into additional languages may improve reporting completeness for the acute POCUS literature.

TRIAL REGISTRATION NUMBER

Open Science Framework Registry (https://osf.io/5pzxs/).

Appraisal of systematic reviews on interventions for postpartum depression: systematic review

BACKGROUND

Postpartum depression (PPD) is a highly prevalent mental health problem that affects parental health with implications for child health in infancy, childhood, adolescence and beyond. The primary aim of this study was to critically appraise available systematic reviews describing interventions for PPD. The secondary aim was to evaluate the methodological quality of the included systematic reviews and their conclusions.

METHODS

An electronic database search of MEDLINE, Embase, and the Cochrane Library from 2000 to 2020 was conducted to identify systematic reviews that examined an intervention for PPD. A Measurement Tool to Assess Systematic Reviews was utilized to independently score each included systematic review which was then critically appraised to better define the most effective therapeutic options for PPD.

RESULTS

Of the 842 studies identified, 83 met the a priori criteria for inclusion. Based on the systematic reviews with the highest methodological quality, we found that use of antidepressants and telemedicine were the most effective treatments for PPD. Symptoms of PPD were also improved by traditional herbal medicine and aromatherapy. Current evidence for physical exercise and cognitive behavioural therapy in treating PPD remains equivocal. A significant, but weak relationship between AMSTAR score and journal impact factor was observed (p = 0.03, r = 0.24; 95% CI, 0.02 to 0.43) whilst no relationship was found between the number of total citations (p = 0.27, r = 0.12; 95% CI, - 0.09 to 0.34), or source of funding (p = 0.19).

CONCLUSION

Overall the systematic reviews on interventions for PPD are of low-moderate quality and are not improving over time. Antidepressants and telemedicine were the most effective therapeutic interventions for PPD treatment.

Reporting and methodological quality of systematic reviews and meta-analysis with protocols in Diabetes Mellitus Type II: A systematic review

BACKGROUND

Systematic reviews with or without meta-analyses (SR/MAs) are strongly encouraged to work from a protocol to facilitate high quality, transparent methodology. The completeness of reporting of a protocol (PRISMA-P) and manuscript (PRISMA) is essential to the quality appraisal (AMSTAR-2) and appropriate use of SR/MAs in making treatment decisions.

OBJECTIVES

The objectives of this study were to describe the completeness of reporting and quality of SR/MAs, assess the correlations between PRISMA-P, PRISMA, and AMSTAR-2, and to identify reporting characteristics between similar items of PRISMA-P and PRISMA.

METHODS

We performed a systematic review of Type 2 Diabetes Mellitus SR/MAs of hypoglycemic agents with publicly available protocols. Cochrane reviews, guidelines, and specific types of MA were excluded. Two reviewers independently, (i) searched PubMed and Embase between 1/1/2015 to 20/3/2019; (ii) identified protocols of included studies by searching the manuscript bibliography, supplementary material, PROSPERO, and Google; (iii) completed PRISMA-P, PRISMA, and AMSTAR-2 tools. Data analysis included descriptive statistics, Pearson correlation, and multivariable linear regression.

RESULTS

Of 357 relevant SR/MAs, 51 had available protocols and were included. The average score for PRISMA-P was 15.8±3.3 (66%; maximum 24) and 25.2±1.1 (93%; maximum 27) for PRISMA. The quality of SR/MAs assessed using the AMSTAR-2 tool identified an overall poor quality (63% critically low, 18% low, 8% moderate, 12% high). The correlation between the PRISMA-P and PRISMA was not significant (r = 0.264; p = 0.06). Correlation was significant between PRISMA-P and AMSTAR-2 (r = 0.333; p = 0.02) and PRISMA and AMSTAR-2 (r = 0.555; p<0.01). Discrepancies in reporting were common between similar PRISMA-P and PRISMA items.

CONCLUSION

Adherence to protocol reporting guidance was poor while manuscript reporting was comprehensive. Protocol completeness is not associated with a completely reported manuscript. Independently, PRISMA-P and PRISMA scores were weakly associated with higher quality assessments but insufficient as a surrogate for quality. Critical areas for quality improvement include protocol description, investigating causes of heterogeneity, and the impact of risk of bias on the evidence synthesis.

The 100 top-cited meta-analyses of diagnostic accuracy in radiology journals: a bibliometric analysis

Purpose

To identify the 100 top-cited meta-analyses of diagnostic accuracy studies published in radiology, medical imaging and nuclear medicine journals.

Methods

A PubMed search with pre-defined criteria was performed. The 100 top-cited articles meta-analyses were retrieved, using a custom Python script and the Scopus Application Programming Interface (Elsevier). Publication, citation and affiliation details were extracted from each meta-analysis. No formal statistical analysis was performed.

Results

The top meta-analysis was cited 394 times, the 100th meta-analysis 38 times. The USA was the top country represented in the papers (33 meta-analyses) followed by The Netherlands, China and Germany. The journal Radiology published 24 studies. The most common modality reported was positron emission tomography (PET) or PET computed tomography (36 instances), followed by magnetic resonance imaging (30 instances) and computed tomography (27 instances). Cardiac (19 meta-analyses), abdominal (18 meta-analyses), followed by neurological (12 meta-analyses) investigations were the most frequently encountered in the top 100 cited meta-analyses.

Conclusions

The 100 top-cited meta-analyses encompass a broad range of imaging modalities and body regions. This may comprise a useful resource for identifying influential evidence-based diagnostic accuracy information in radiology.

The Hong Kong Principles for assessing researchers: Fostering research integrity

For knowledge to benefit research and society, it must be trustworthy. Trustworthy research is robust, rigorous, and transparent at all stages of design, execution, and reporting. Assessment of researchers still rarely includes considerations related to trustworthiness, rigor, and transparency. We have developed the Hong Kong Principles (HKPs) as part of the 6th World Conference on Research Integrity with a specific focus on the need to drive research improvement through ensuring that researchers are explicitly recognized and rewarded for behaviors that strengthen research integrity. We present five principles: responsible research practices; transparent reporting; open science (open research); valuing a diversity of types of research; and recognizing all contributions to research and scholarly activity. For each principle, we provide a rationale for its inclusion and provide examples where these principles are already being adopted.

Research Pearls: Checklists and Flowcharts to Improve Research Quality

To instill quality in published clinical research, reporting guidelines, consisting of checklists and flowcharts, were developed to protect against reporting poorly designed research, and researchers should be aware of the available instruments and their appropriate use. With the popularity of synthetic reviews, meta-analyses, and systematic reviews, there is a greater need to assess risk of bias and study quality. This review highlights the most frequently used guidelines and checklists, risk-of-bias scales, and quality rating scales that can assist researchers with improving their research and its eventual publication.

The Hong Kong Principles for assessing researchers: Fostering research integrity

For knowledge to benefit research and society, it must be trustworthy. Trustworthy research is robust, rigorous, and transparent at all stages of design, execution, and reporting. Assessment of researchers still rarely includes considerations related to trustworthiness, rigor, and transparency. We have developed the Hong Kong Principles (HKPs) as part of the 6th World Conference on Research Integrity with a specific focus on the need to drive research improvement through ensuring that researchers are explicitly recognized and rewarded for behaviors that strengthen research integrity. We present five principles: responsible research practices; transparent reporting; open science (open research); valuing a diversity of types of research; and recognizing all contributions to research and scholarly activity. For each principle, we provide a rationale for its inclusion and provide examples where these principles are already being adopted.

Overinterpretation of Research Findings: Evaluation of “Spin” in Systematic Reviews of Diagnostic Accuracy Studies in High–Impact Factor Journals

Background

To compare the frequency of “spin” in systematic reviews of diagnostic accuracy studies in high-impact journals with the frequency a previously assessed series of reviews.

Methods

Medline was searched from January 2010 to January 2019. Systematic reviews of diagnostic accuracy studies were included if they reported a meta-analysis and were published in a journal with an impact factor &gt;5. Two investigators independently scored each included systematic review for positivity of conclusions and for actual and potential overinterpretation practices.

Results

Of 137 included systematic reviews, actual overinterpretation was present in ≥1 form in the abstract in 63 (46%) and in the full-text report in 52 (38%); 108 (79%) contained a form of potential overinterpretation. Compared with the previously assessed series (reviews published 2015–2016), reviews in this series were less likely to contain ≥1 form of actual overinterpretation in the abstract and full-text report or ≥1 form of potential overinterpretation (P &lt; 0.001 for all comparisons). The significance of these comparisons did not persist for actual overinterpretation in sensitivity analysis in which Cochrane systematic reviews were removed. Reviews published in the Cochrane Database of Systematic Reviews were less likely to contain actual overinterpretation in the abstract or the full-text report than reviews in other high-impact journals (P &lt; 0.001 for both comparisons).

Conclusions

Reviews of diagnostic accuracy studies in high-impact journals are less likely to contain overinterpretation or spin. This difference is largely due to the reviews published in the Cochrane Database of Systematic Reviews, which contain spin less often than reviews published in other high-impact journals.

Mapping the characteristics of meta-analyses of pharmacy services: a systematic review

BACKGROUND

Suboptimal meta-analyses with misleading conclusions are frequently published in the health areas, and they can compromise decision making in clinical practice.

AIM OF THE REVIEW

This systematic review aimed to map the characteristics of published meta-analyses of pharmacy services and their association with the study conclusions.

METHOD

We searched electronic databases (PubMed, Scopus, and Web of Science) to identify published meta-analyses of pharmacy services up to January 2019. Components of meta-analyses were extracted (i.e. studies' metadata; methods used in the systematic review; description of the statistical model used for the meta-analysis; main results; conflict of interest and funding source). The methodological quality was evaluated using the R-AMSTAR tool.

RESULTS

A total of 85 meta-analyses were included, with 2016 as the median publication year. Overall, the methodological quality of meta-analyses of pharmacy services was considered suboptimal. Only one-third of authors registered a protocol; complete search strategy and raw data were provided by 55.3% and 9.4% of studies, respectively. Evidence strength (GRADE) was evaluated in only 19.2% of studies. PRISMA and Cochrane recommendations were stated to be followed in 60% and 27.4% of articles, respectively. Around half of studies performed sensitivity analysis, however, the prediction interval was presented by only one meta-analysis. Studies that favoured the pharmacists' interventions poorly discussed the methodological quality and heterogeneity of primary trials.

CONCLUSION

Poor conduction and reporting were observed in meta-analyses of pharmacy services, especially in those that favoured the pharmacist's interventions. Reproducibility and transparency should be rigorously ensured by journal editors and peer-reviewers.

Adherence to the Standards for Reporting of Diagnostic Accuracy (STARD) 2015 Guidelines in Acute Point-of-Care Ultrasound Research

IMPORTANCE

Incomplete reporting of diagnostic accuracy research impairs assessment of risk of bias and limits generalizability. Point-of-care ultrasound has become an important diagnostic tool for acute care physicians, but studies assessing its use are of varying methodological quality.

OBJECTIVE

To assess adherence to the Standards for Reporting of Diagnostic Accuracy (STARD) 2015 guidelines in the literature on acute care point-of-care ultrasound.

EVIDENCE REVIEW

MEDLINE was searched to identify diagnostic accuracy studies assessing point-of-care ultrasound published in critical care, emergency medicine, or anesthesia journals from 2016 to 2019. Studies were evaluated for adherence to the STARD 2015 guidelines, with the following variables analyzed: journal, country, STARD citation, STARD-adopting journal, impact factor, patient population, use of supplemental material, and body region. Data analysis was performed in November 2019.

FINDINGS

Seventy-four studies were included in this systematic review for assessment. Overall adherence to STARD was moderate, with 66% (mean [SD], 19.7 [2.9] of 30 items) of STARD items reported. Items pertaining to imaging specifications, patient population, and readers of the index test were frequently reported (>66% of studies). Items pertaining to blinding of readers to clinical data and to the index or reference standard, analysis of heterogeneity, indeterminate and missing data, and time intervals between index and reference test were either moderately (33%-66%) or infrequently (<33%) reported. Studies in STARD-adopting journals (mean [SD], 20.5 [2.9] items in adopting journals vs 18.6 [2.3] items in nonadopting journals; P = .002) and studies citing STARD (mean [SD], 21.3 [0.9] items in citing studies vs 19.5 [2.9] items in nonciting studies; P = .01) reported more items. Variation by country and journal of publication were identified. No differences in STARD adherence were identified by body region imaged (mean [SD], abdominal, 20.0 [2.5] items; head and neck, 17.8 [1.6] items; musculoskeletal, 19.2 [3.1] items; thoracic, 20.2 [2.8] items; and other or procedural, 19.8 [2.7] items; P = .29), study design (mean [SD], prospective, 19.7 [2.9] items; retrospective, 19.7 [1.8] items; P > .99), patient population (mean [SD], pediatric, 20.0 [3.1] items; adult, 20.2 [2.7] items; mixed, 17.9 [1.9] items; P = .09), use of supplementary materials (mean [SD], yes, 19.2 [3.0] items; no, 19.7 [2.8] items; P = .91), or journal impact factor (mean [SD], higher impact factor, 20.3 [3.1] items; lower impact factor, 19.1 [2.4] items; P = .08).

CONCLUSIONS AND RELEVANCE

Overall, the literature on acute care point-of-care ultrasound showed moderate adherence to the STARD 2015 guidelines, with more complete reporting found in studies citing STARD and those published in STARD-adopting journals. This study has established a current baseline for reporting; however, future studies are required to understand barriers to complete reporting and to develop strategies to mitigate them.

Searching practices and inclusion of unpublished studies in systematic reviews of diagnostic accuracy

Introduction

Many diagnostic accuracy studies are never reported in full in a peer‐reviewed journal. Searching for unpublished studies may avoid bias due to selective publication, enrich the power of systematic reviews, and thereby help to reduce research waste. We assessed searching practices among recent systematic reviews of diagnostic accuracy.

Methods

We extracted data from 100 non‐Cochrane systematic reviews of diagnostic accuracy indexed in MEDLINE and published between October 2017 and January 2018 and from all 100 Cochrane systematic reviews of diagnostic accuracy published by December 2018, irrespective of whether meta‐analysis had been performed.

Results

Non‐Cochrane and Cochrane reviews searched a median of 4 (IQR 3‐5) and 6 (IQR 5‐9) databases, respectively; most often MEDLINE/PubMed (n = 100 and n = 100) and EMBASE (n = 81 and n = 100). Additional efforts to identify studies beyond searching bibliographic databases were performed in 76 and 98 reviews, most often through screening reference lists (n = 71 and n = 96), review/guideline articles (n = 18 and n = 52), or citing articles (n = 3 and n = 42). Specific sources of unpublished studies were searched in 22 and 68 reviews, for example, conference proceedings (n = 4 and n = 18), databases only containing conference abstracts (n = 2 and n = 33), or trial registries (n = 12 and n = 39). At least one unpublished study was included in 17 and 23 reviews. Overall, 39 of 2082 studies (1.9%) included in non‐Cochrane reviews were unpublished, and 64 of 2780 studies (2.3%) in Cochrane reviews, most often conference abstracts (97/103).

Conclusion

Searching practices vary considerably across systematic reviews of diagnostic accuracy. Unpublished studies are a minimal fraction of the evidence included in recent reviews.

Evidence-Based Medicine in the Domain of Nuclear Medicine, the Fifty-Year View

The discipline of evidence-based medicine (EBM), though yet unnamed, was in its infancy when Seminars in Nuclear Medicine was inaugurated in 1971; commemorating the golden anniversary of this prestigious journal and the contemporaneous reign of its editors by publishing a 50-year historical consideration of EBM seems most apropos. EBM should be understood as a system of methods to eliminate partiality and improve the quality of evidence in the performance and review of data; much of EBM revolves around ensuring that conclusions are derived from rigorous research studies that protect against bias and are widely generalizable to other groups of patients. Subdomains within EBM that we will survey in this review include methods of performing and evaluating primary studies, standards of reporting of medical studies, methods of combining and aggregating data, and finally, methods of creating clinical practice guidelines. While many practitioners of nuclear medicine may not presently be familiar with the innovations of EBM, having been introduced after their formal education was completed, with the eventual arrival of more-recently trained staff, firm recommendations from our primary research journals, and with efforts to educate practicing physicians, this shortcoming is being addressed.

Reporting quality and statistical analysis of published dose-response meta-analyses was suboptimal: a cross-sectional literature survey

OBJECTIVES

The objective of this study was to investigate the characteristics, methodological quality, and reporting of statistical analyses of published dose-response meta-analyses (DRMAs).

STUDY DESIGN AND SETTING

We searched PubMed to identify DRMAs published in 2017. The reporting characteristics and methodological qualities were assessed by the PRISMA (27 items) and AMSTAR (11 items), respectively. We also summarized the reporting of statistical analyses of included DRMAs.

RESULTS

We identified 93 DRMAs, most of which (59/93) were conducted by Chinese researchers and the main outcome was the incidence of cancers. Of the PRISMA and AMSTAR items, twenty and five were well complied (80% or more), respectively. The compliance rates of several PRISMA checklist items, such as structured summary, objectives, protocol and registration, and funding, were less than 50%. There were no criteria to estimate the doses for the open-ended intervals of exposure or intervention doses. When the restricted cubic splines were used to fit nonlinear dose-response relationships, there were also no criteria to determine the fixed knots.

CONCLUSION

The adherence to the methodological items of reporting guidelines and statistical analysis of published DRMAs were suboptimal. Development of reporting guidelines to assist authors in writing and readers in critically appraising the reports of DRMAs is timely.

Steps toward more complete reporting of systematic reviews of diagnostic test accuracy: Preferred Reporting Items for Systematic Reviews and Meta-Analyses of Diagnostic Test Accuracy (PRISMA-DTA)

Reporting standards in biomedical research have been shown to be suboptimal. The publication of the PRISMA statement has improved the completeness of reporting of systematic reviews, but several issues specific to diagnostic test accuracy are not included in the PRISMA statement. Therefore, a diagnostic test accuracy extension of the PRISMA statement, PRISMA-DTA, was created. This commentary addresses completeness of reporting in systematic reviews, the PRISMA-DTA statement, and strategies for optimal uptake of reporting guidelines.

Meta-analyses indexed in PsycINFO had a better completeness of reporting when they mention PRISMA

OBJECTIVES

To investigate the effect of the explicit mention of PRISMA, a statement designed to help authors report meta-analyses (MAs), on the reporting completeness of MAs.

STUDY DESIGN AND SETTING

Two investigators evaluated a random sample of 206 MAs indexed in PsycINFO in 2016; 100 explicitly mentioned PRISMA and 106 did not. Two authors independently evaluated the 27 PRISMA items and extracted factors that could potentially be associated with reporting completeness. The data were analyzed descriptively.

RESULTS

Among our 206 MAs, perfect adherence to PRISMA was found in less than 4%, of which 87% explicitly followed PRISMA. The following items were encountered significantly more frequently in MAs that explicitly mentioned PRISMA than in those that did not: summary, protocol, information sources, search strategy, study characteristics, results of individual studies, funding, study selection, risk of bias in individual studies, and bias across studies. The journal's impact factor, endorsement of PRISMA by the journal, number of authors, country of author, open access, and design of the included studies were significantly and positively associated with the explicit mention of PRISMA.

CONCLUSIONS

Even if far from optimal, the explicit mention of PRISMA has a positive influence on the reporting completeness of MAs from PsycINFO.

Reporting quality of systematic reviews of interventions aimed at improving vaccination coverage: compliance with PRISMA guidelines

Systematic reviews have become increasingly important for informing clinical practice and policy; however, little is known about the reporting characteristics and quality of SRs of interventions to improve immunization coverage in different settings. The aim of this study was to assess the reporting quality of systematic reviews of interventions aimed at improving vaccination coverage using the recommended Preferred Reporting Items for Systematic Reviews and Meta-analysis (PRISMA) guideline.PubMed and Cochrane Library were searched to identify SRs of interventions to improve immunization coverage, indexed up to May 2016. Two authors independently screened the search output, assessed study eligibility, and extracted data from eligible SRs using a 27-item data collection form derived from PRISMA. Discrepancies in reviews assessments were resolved by discussion and consensus.A total of 57 reviews were included in this study with a mean percentage of applicable PRISMA items that were met across all studies of 66% (range 19-100%) and median compliance of 70%. 39 out of the 57 reviews were published after the release of the PRISMA statement in 2009. Highest compliance was observed in items related to the "description of rational", "description of eligibility criteria", "synthesis of results" and "provision of a general interpretation of the results" (items #3, #6, #14 and #26, respectively). Compliance was poorest in the items "describing summary of evidence" (item 24, 19%), "describing indication of review protocol and registration" (item 5, 26%) and "describing results of risk of bias across studies (item 22, 33%).The overall reporting quality of systematic reviews of interventions to improve vaccination coverage requires significant improvement. There remains a need for additional research targeted at addressing potential barriers to compliance and strategies to improve compliance with PRISMA guideline.

Scoping review on interventions to improve adherence to reporting guidelines in health research

OBJECTIVES

The goal of this study is to identify, analyse and classify interventions to improve adherence to reporting guidelines in order to obtain a wide picture of how the problem of enhancing the completeness of reporting of biomedical literature has been tackled so far.

DESIGN

Scoping review.

SEARCH STRATEGY

We searched the MEDLINE, EMBASE and Cochrane Library databases and conducted a grey literature search for (1) studies evaluating interventions to improve adherence to reporting guidelines in health research and (2) other types of references describing interventions that have been performed or suggested but never evaluated. The characteristics and effect of the evaluated interventions were analysed. Moreover, we explored the rationale of the interventions identified and determined the existing gaps in research on the evaluation of interventions to improve adherence to reporting guidelines.

RESULTS

109 references containing 31 interventions (11 evaluated) were included. These were grouped into five categories: (1) training on the use of reporting guidelines, (2) improving understanding, (3) encouraging adherence, (4) checking adherence and providing feedback, and (5) involvement of experts. Additionally, we identified lack of evaluated interventions (1) on training on the use of reporting guidelines and improving their understanding, (2) at early stages of research and (3) after the final acceptance of the manuscript.

CONCLUSIONS

This scoping review identified a wide range of strategies to improve adherence to reporting guidelines that can be taken by different stakeholders. Additional research is needed to assess the effectiveness of many of these interventions.

Characteristics, trend, and methodological quality of systematic reviews and meta-analyses in nuclear medicine: A bibliometric analysis of studies published between 2005 and 2016

To evaluate the characteristics, trend, and quality of systematic reviews and meta-analyses in nuclear medicine.We performed a PubMed search to identify systematic reviews and meta-analyses published between 2005 and 2016 in the field of nuclear medicine. The following data were extracted: journal name, impact factor, type of study, topics with cancer type, imaging modalities, authors (number, country, affiliation, presence of nuclear medicine specialists and statisticians, discordance between the first and corresponding authors), funding, methodological quality, methods used for quality assessment, and statistical methods.We included 185 nuclear medicine articles. Meta-analyses (n = 164; 88.6%) were published about 7 times more frequently than systematic reviews. Oncology was the most commonly studied topic (n = 125, 67.6%). The first authors were most frequently located in China (n = 73; 39.5%). PET was the most commonly used modality (n = 150; 81.1%). Both the number of authors and the ratio of discordance between the first and corresponding authors tended to progressively increase over time.The mean AMSTAR score increased over time (5.77 in 2005-2008, 6.71 in 2009-2012, and 7.44 in 2013-2016). The proportion of articles with quality assessment increased significantly (20/26 in 2005-2008, 54/65 in 2009-2012, and 79/94 in 2013-2016). The most commonly used assessment tool was quality assessment of diagnostic accuracy studies (n = 85; 54.9%).The number and quality of systematic reviews and meta-analyses in nuclear medicine have significantly increased over the review period; however, the quality of these articles varies. Efforts to overcome specific weaknesses of the methodologies can provide opportunities for quality improvement.

Perception of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement of authors publishing reviews in nursing journals: a cross-sectional online survey

OBJECTIVE

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement has been developed as a guideline for reporting systematic reviews and meta-analyses. Despite the prevalent use of the PRISMA statement in medicine and nursing, no studies have examined authors' perception of it. The purpose of this study is to explore the perception of the PRISMA statement of authors who published reviews, meta-analyses, or both in nursing journals.

DESIGN

Cross-sectional descriptive study.

METHODS

An online survey was conducted among authors who published reviews, meta-analyses, or both in nursing journals between 2011 and 2017. The selected authors' email addresses were extracted from the PUBMED database. A questionnaire-with a 10-point Likert scale (1-not important at all to 10-very important)-was developed to elicit their responses regarding their perception of not only the PRISMA statement as a whole, but also the individual items therein.

RESULTS

Invitations were sent to 1960 valid email addresses identified, with 230 responses (response rate: 11.7%) and 181 completed responses (completion rate: 9.2%). The average perceived importance of the PRISMA statement was 8.66 (SD=1.35), while the perceived importance for the individual items ranged from 7.74 to 9.32. Six items were rated significantly higher than the average rating, whereas one item was rated significantly lower.

CONCLUSION

Most respondents perceived the PRISMA statement as important. Items related to information sources, selection, search-flow presentation, summary of findings, limitations and interpretation were deemed more important while the registration was deemed less so.

Replicate systematic review and meta-analyses on robotic surgery: a quality appraisal and overlap investigation

BACKGROUND

The number of publications of systematic reviews and meta-analyses (MAs) on robotic surgery have been increasing, including many investigating the same topic. Their quality and extent of overlap remains unclear. We assessed the quality of the MAs in this area and investigated the extent of their overlap.

METHODS

Relevant studies were identified by searching the MEDLINE, EMBASE, and Cochrane Library databases up to August 1, 2017. Reporting and methodological quality levels were assessed using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) and Assessment of Multiple Systematic Reviews (AMSTAR) checklists. A thorough investigation of the extent of overlap was performed.

RESULTS

In total, 90 MAs in 5 surgical subspecialties were included after full-text review. The mean reporting and methodological quality scores were 22.5 (83.2%) and 7.6 (69.2%), respectively. Authors from university-affiliated institutions and the presence of statistician or epidemiologist coauthors were associated with better-reporting quality scores. The topics with the most overlapping MAs (all ≥ 6) were robot-assisted thyroidectomy, prostatectomy, gastrectomy, colectomy, and fundoplication. 36 (40%) of the included MAs cited previous MAs on the same topic. Among the 7 MAs comparing robot-assisted radical prostatectomy to the open procedure, most (6/7) drew the same conclusion. 50 to 86% of MAs on this topic included the same trials as primary studies.

CONCLUSION

Conducting multiple overlapping MAs with identical conclusions on the same topic that are of suboptimal quality may be a waste of resource and effort. Authors from university-affiliated institutes and experts in epidemiology and statistics are more likely to conduct MAs that have better quality. More guidelines and registries are needed to avoid overlapping MAs.