Has the quality of reporting improved since it became mandatory to use the Standards for Reporting Diagnostic Accuracy?

OBJECTIVES

To investigate whether making the Standards for Reporting Diagnostic Accuracy (STARD) mandatory by the leading journal 'Radiology' in 2016 improved the quality of reporting of diagnostic accuracy studies.

METHODS

A validated search term was used to identify diagnostic accuracy studies published in Radiology in 2015 and 2019. STARD adherence was assessed by two independent reviewers. Each item was scored as yes (1 point) if adequately reported or as no (0 points) if not. The total STARD score per article was calculated. Wilcoxon-Mann-Whitney tests were used to evaluate differences of the total STARD scores between 2015 and 2019. In addition, the total STARD score was compared between studies stratified by study design, citation rate, and data collection.

RESULTS

The median number of reported STARD items for the total of 66 diagnostic accuracy studies from 2015 to 2019 was 18.5 (interquartile range [IQR] 17.5-20.0) of 29. Adherence to the STARD checklist significantly improved the STARD score from a median of 18.0 (IQR 15.5-19.5) in 2015 to a median of 19.5 (IQR 18.5-21.5) in 2019 (p < 0.001). No significant differences were found between studies stratified by mode of data collection (prospective vs. retrospective studies, p = 0.68), study design (cohort vs. case-control studies, p = 0.81), and citation rate (two groups divided by median split [< 0.56 citations/month vs. ≥ 0.56 citations/month], p = 0.54).

CONCLUSIONS

Making use of the STARD checklist mandatory significantly increased the adherence with reporting standards for diagnostic accuracy studies and should be considered by editors and publishers for widespread implementation.

CRITICAL RELEVANCE STATEMENT

Editors may consider making reporting guidelines mandatory to improve the scientific quality.

Transparent transdisciplinary reporting in geriatric research using the EQUATOR Network

This editorial comments on the article by Carpenter et al. in this issue.

Determinants of societal and academic recognition: Evidence from randomised controlled trials

Given the increasing importance of recognition in academia and the vital role of randomised controlled trials (RCTs) in medical research and clinical decisions, this study verifies how RCTs’ academic and societal impacts are affected by visibility factors, subjects and methodological validity. This study concentrated on a sample of 446 RCTs indexed in Scopus and evaluated by Cochrane reviewers in terms of their methodological validity. The altmetrics, bibliometric and bibliographical information were extracted from Altmetric.com and Scopus, and the contributing countries’ development ranks were obtained from the United Nations Development report. The linear regression analyses revealed that citations and altmetrics depend on some subjects. They are also affected by publication year and journals’ previous reputation. Citations are also affected by keyword counts and reference counts. Keyword counts and contributing countries’ developmental rank also predict the tweet counts. While none of the methodological validity dimensions were found to predict citations, ‘Incomplete Outcome Data’ and ‘Random Sequence Generation’ significantly, though slightly, affect Mendeley Readership and tweets, respectively. By confirming the dependence of RCTs’ recognition on some methodological validity features and attention-inducing characteristics, the study provides further evidence on the interaction of quality and visibility dynamisms in the recognition network and the complementary role of societal mentions for academic citation.

Citation impact was highly variable for reporting guidelines of health research: a citation analysis

OBJECTIVES

Over 400 reporting guidelines are currently published, but the frequency of their use by authors to accurately and transparently report research remains unclear. This study examined citation counts of reporting guidelines and characteristics contributing to their citation impact.

STUDY DESIGN AND SETTING

Web of Science database was searched for citation counts of all reporting guidelines with a minimum citation age of 5 years. The total citation impact, mean citation impact and the factors contributing to 2- and 5-year citation rate were established.

RESULTS

The search identified 296 articles of reporting guidelines from 1995 to 2013. The mean citations per year was 32.4 (95% confidence interval, 22.3-42.4 citations). The factors associated with 2- and 5-year citation performance of reporting guidelines included the following: open access to the reporting guideline, field of the publishing journal (general vs. specialized medical journal), impact factor of the publishing journal, simultaneous publication in multiple journals, and a male first author.

CONCLUSION

The citation rate across reporting guidelines varied with journal impact factor, open access publication, field of the publishing journal, simultaneous publications, and a male first author. Gaps in citations highlight opportunities to increase visibility and encourage author use of reporting guidelines.

Adherence to the Standards for Reporting of Diagnostic Accuracy (STARD) 2015 Guidelines in Acute Point-of-Care Ultrasound Research

IMPORTANCE

Incomplete reporting of diagnostic accuracy research impairs assessment of risk of bias and limits generalizability. Point-of-care ultrasound has become an important diagnostic tool for acute care physicians, but studies assessing its use are of varying methodological quality.

OBJECTIVE

To assess adherence to the Standards for Reporting of Diagnostic Accuracy (STARD) 2015 guidelines in the literature on acute care point-of-care ultrasound.

EVIDENCE REVIEW

MEDLINE was searched to identify diagnostic accuracy studies assessing point-of-care ultrasound published in critical care, emergency medicine, or anesthesia journals from 2016 to 2019. Studies were evaluated for adherence to the STARD 2015 guidelines, with the following variables analyzed: journal, country, STARD citation, STARD-adopting journal, impact factor, patient population, use of supplemental material, and body region. Data analysis was performed in November 2019.

FINDINGS

Seventy-four studies were included in this systematic review for assessment. Overall adherence to STARD was moderate, with 66% (mean [SD], 19.7 [2.9] of 30 items) of STARD items reported. Items pertaining to imaging specifications, patient population, and readers of the index test were frequently reported (>66% of studies). Items pertaining to blinding of readers to clinical data and to the index or reference standard, analysis of heterogeneity, indeterminate and missing data, and time intervals between index and reference test were either moderately (33%-66%) or infrequently (<33%) reported. Studies in STARD-adopting journals (mean [SD], 20.5 [2.9] items in adopting journals vs 18.6 [2.3] items in nonadopting journals; P = .002) and studies citing STARD (mean [SD], 21.3 [0.9] items in citing studies vs 19.5 [2.9] items in nonciting studies; P = .01) reported more items. Variation by country and journal of publication were identified. No differences in STARD adherence were identified by body region imaged (mean [SD], abdominal, 20.0 [2.5] items; head and neck, 17.8 [1.6] items; musculoskeletal, 19.2 [3.1] items; thoracic, 20.2 [2.8] items; and other or procedural, 19.8 [2.7] items; P = .29), study design (mean [SD], prospective, 19.7 [2.9] items; retrospective, 19.7 [1.8] items; P > .99), patient population (mean [SD], pediatric, 20.0 [3.1] items; adult, 20.2 [2.7] items; mixed, 17.9 [1.9] items; P = .09), use of supplementary materials (mean [SD], yes, 19.2 [3.0] items; no, 19.7 [2.8] items; P = .91), or journal impact factor (mean [SD], higher impact factor, 20.3 [3.1] items; lower impact factor, 19.1 [2.4] items; P = .08).

CONCLUSIONS AND RELEVANCE

Overall, the literature on acute care point-of-care ultrasound showed moderate adherence to the STARD 2015 guidelines, with more complete reporting found in studies citing STARD and those published in STARD-adopting journals. This study has established a current baseline for reporting; however, future studies are required to understand barriers to complete reporting and to develop strategies to mitigate them.

Has the Objective Quality of Evidence in Imaging Papers Changed Over the Last 20 Years?

RATIONALE AND OBJECTIVES

We aimed to determine if both evidence level (EL) as well as clinical efficacy (CE) of imaging manuscripts have changed over the last 20 years.

MATERIALS AND METHODS

With our review of medical literature, Institutional Review Board approval was waived, and no informed consent was required. Using Web of Science, we determined the 10 highest impact factor imaging journals. For each journal the 10 most cited and 10 average cited papers were compared for the following years: 1994, 1998, 2002, 2006, 2010, and 2014. EL was graded using the same criteria as the Journal of Bone and Joint Surgery (Wright et al., 2003). CE was graded using the criteria of Thornbury and Fryback (1991). Statistical software R and package lme4 were used to fit mixed regression models with fixed effects for group, year, and a random effect for journal.

RESULTS

EL has improved -0.03 every year on average (P < .001). The more cited papers had better ELs (group effect = -0.23, SE 0.09, P = .011). CE is lower in top cited compared to average cited articles, although the differences were not statistically significant (group effect = -0.14, SE = 0.09, P = .16). CE level increased modestly in both groups over this 20-year time period (0.06 per year, SE = 0.007, P < .001).

CONCLUSION

Over the last 20 years, imaging journal articles have improved modestly in quality of evidence, as measured by EL and CE.

Preanalytical investigations of phlebotomy: methodological aspects, pitfalls and recommendations

Phlebotomy is often addressed as a crucial process in the pre-analytical phase, in which a large part of laboratory errors take place, but to date there is not yet a consolidated methodological paradigm. Seeking literature, we found 36 suitable investigations issued between 1996 and 2016 (April) dealing with the investigation of pre-analytical factors related to phlebotomy. We found that the largest part of studies had a cohort of healthy volunteers (22/36) or outpatients (11/36), with the former group showing a significantly smaller median sample size (N = 20, IQR: 17.5-30 and N = 88, IQR: 54.5-220.5 respectively, P < 0.001). Moreover, the largest part investigated one pre-analytical factor (26/36) and regarded more than one laboratory test (29/36), and authors preferably used paired Student’s t-test (17/36) or Wilcoxon’s test (11/36), but calibration (i.e. sample size calculation for a detectable effect) was addressed only in one manuscript. The Bland-Altman plot was often the preferred method used to estimate bias (12/36), as well as the Passing-Bablok regression for agreement (8/36). However, often papers did assess neither bias (12/36) nor agreement (24/36). Clinical significance of bias was preferably assessed comparing to a database value (16/36), and it resulted uncorrelated with the size of the effect produced by the factor (P = 0.142). However, the median effect size (ES) resulted significantly larger if the associated factor was clinically significant instead of non-significant (ES = 1.140, IQR: 0.815-1.700 and ES = 0.349, IQR: 0.228-0.531 respectively, P < 0.001). On these evidences, we discussed some recommendations for improving methodological consistency, delivering reliable results, as well as ensuring accessibility to practical evidences.

Does the Reporting Quality of Diagnostic Test Accuracy Studies, as Defined by STARD 2015, Affect Citation?

Objective

To determine the rate with which diagnostic test accuracy studies that are published in a general radiology journal adhere to the Standards for Reporting of Diagnostic Accuracy Studies (STARD) 2015, and to explore the relationship between adherence rate and citation rate while avoiding confounding by journal factors.

Materials and Methods

All eligible diagnostic test accuracy studies that were published in the Korean Journal of Radiology in 2011–2015 were identified. Five reviewers assessed each article for yes/no compliance with 27 of the 30 STARD 2015 checklist items (items 28, 29, and 30 were excluded). The total STARD score (number of fulfilled STARD items) was calculated. The score of the 15 STARD items that related directly to the Quality Assessment of Diagnostic Accuracy Studies (QUADAS)-2 was also calculated. The number of times each article was cited (as indicated by the Web of Science) after publication until March 2016 and the article exposure time (time in months between publication and March 2016) were extracted.

Results

Sixty-three articles were analyzed. The mean (range) total and QUADAS-2-related STARD scores were 20.0 (14.5–25) and 11.4 (7–15), respectively. The mean citation number was 4 (0–21). Citation number did not associate significantly with either STARD score after accounting for exposure time (total score: correlation coefficient = 0.154, p = 0.232; QUADAS-2-related score: correlation coefficient = 0.143, p = 0.266).

Conclusion

The degree of adherence to STARD 2015 was moderate for this journal, indicating that there is room for improvement. When adjusted for exposure time, the degree of adherence did not affect the citation rate.