Small Study Effects in Diagnostic Imaging Accuracy: A Meta-Analysis

Integrity of Clinical Neuroradiological Research

PURPOSE

It is unclear if undesired practices such as scientific fraud, publication bias, and honorary authorship are present in neuroradiology. Therefore, the objective was to explore the integrity of clinical neuroradiological research using a survey method.

METHODS

Corresponding authors who published in one of four top clinical neuroradiology journals were invited to complete a survey about integrity in clinical neuroradiology research.

RESULTS

A total of 232 corresponding authors participated in our survey. Confidence in the integrity of published scientific work in clinical neuroradiology (0-10 point scale) was rated as a median score of 8 (range 3-10). In linear regression analysis, respondents from Asia had significantly higher confidence (beta coefficient of 0.569, 95% confidence interval, CI: 0.049-1.088, P = 0.032). Of the respondents 8 (3.4%) reported to have committed scientific fraud in the past 5 years, whereas 66 respondents (28.4%) reported to have witnessed or suspected scientific fraud by anyone from their department in the past 5 years. A total of 192 respondents (82.8%) thought that a study with positive results is more likely to be accepted by a journal than a similar study with negative results and 96 respondents (41.4%) had an honorary author on any of their publications in the past 5 years.

CONCLUSION

Experts in the field have overall high confidence in published clinical neuroradiology research; however, scientific integrity concerns are not negligible, publication bias is a problem and honorary authorship is common. The findings from this survey may help to increase awareness and vigilance among anyone involved in clinical neuroradiological research.

Image-based artificial intelligence for the prediction of pathological complete response to neoadjuvant chemoradiotherapy in patients with rectal cancer: a systematic review and meta-analysis

OBJECTIVE

Artificial intelligence (AI) holds enormous potential for noninvasively identifying patients with rectal cancer who could achieve pathological complete response (pCR) following neoadjuvant chemoradiotherapy (nCRT). We aimed to conduct a meta-analysis to summarize the diagnostic performance of image-based AI models for predicting pCR to nCRT in patients with rectal cancer.

METHODS

This study followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. A literature search of PubMed, Embase, Cochrane Library, and Web of Science was performed from inception to July 29, 2023. Studies that developed or utilized AI models for predicting pCR to nCRT in rectal cancer from medical images were included. The Quality Assessment of Diagnostic Accuracy Studies-AI was used to appraise the methodological quality of the studies. The bivariate random-effects model was used to summarize the individual sensitivities, specificities, and areas-under-the-curve (AUCs). Subgroup and meta-regression analyses were conducted to identify potential sources of heterogeneity. Protocol for this study was registered with PROSPERO (CRD42022382374).

RESULTS

Thirty-four studies (9933 patients) were identified. Pooled estimates of sensitivity, specificity, and AUC of AI models for pCR prediction were 82% (95% CI: 76-87%), 84% (95% CI: 79-88%), and 90% (95% CI: 87-92%), respectively. Higher specificity was seen for the Asian population, low risk of bias, and deep-learning, compared with the non-Asian population, high risk of bias, and radiomics (all P < 0.05). Single-center had a higher sensitivity than multi-center (P = 0.001). The retrospective design had lower sensitivity (P = 0.012) but higher specificity (P < 0.001) than the prospective design. MRI showed higher sensitivity (P = 0.001) but lower specificity (P = 0.044) than non-MRI. The sensitivity and specificity of internal validation were higher than those of external validation (both P = 0.005).

CONCLUSIONS

Image-based AI models exhibited favorable performance for predicting pCR to nCRT in rectal cancer. However, further clinical trials are warranted to verify the findings.

Heterogeneity in Systematic Reviews of Medical Imaging Diagnostic Test Accuracy Studies: A Systematic Review

Importance

Systematic reviews of medical imaging diagnostic test accuracy (DTA) studies are affected by between-study heterogeneity due to a range of factors. Failure to appropriately assess the extent and causes of heterogeneity compromises the interpretability of systematic review findings.

Objective

To assess how heterogeneity has been examined in medical imaging DTA studies.

Evidence Review

The PubMed database was searched for systematic reviews of medical imaging DTA studies that performed a meta-analysis. The search was limited to the 40 journals with highest impact factor in the radiology, nuclear medicine, and medical imaging category in the InCites Journal Citation Reports of 2021 to reach a sample size of 200 to 300 included studies. Descriptive analysis was performed to characterize the imaging modality, target condition, type of meta-analysis model used, strategies for evaluating heterogeneity, and sources of heterogeneity identified. Multivariable logistic regression was performed to assess whether any factors were associated with at least 1 source of heterogeneity being identified in the included meta-analyses. Methodological quality evaluation was not performed. Data analysis occurred from October to December 2022.

Findings

A total of 242 meta-analyses involving a median (range) of 987 (119-441 510) patients across a diverse range of disease categories and imaging modalities were included. The extent of heterogeneity was adequately described (ie, whether it was absent, low, moderate, or high) in 220 studies (91%) and was most commonly assessed using the I2 statistic (185 studies [76%]) and forest plots (181 studies [75%]). Heterogeneity was rated as moderate to high in 191 studies (79%). Of all included meta-analyses, 122 (50%) performed subgroup analysis and 87 (36%) performed meta-regression. Of the 242 studies assessed, 189 (78%) included 10 or more primary studies. Of these 189 studies, 60 (32%) did not perform meta-regression or subgroup analysis. Reasons for being unable to investigate sources of heterogeneity included inadequate reporting of primary study characteristics and a low number of included primary studies. Use of meta-regression was associated with identification of at least 1 source of variability (odds ratio, 1.90; 95% CI, 1.11-3.23; P = .02).

Conclusions and Relevance

In this systematic review of assessment of heterogeneity in medical imaging DTA meta-analyses, most meta-analyses were impacted by a moderate to high level of heterogeneity, presenting interpretive challenges. These findings suggest that, despite the development and availability of more rigorous statistical models, heterogeneity appeared to be incomplete, inconsistently evaluated, or methodologically questionable in many cases, which lessened the interpretability of the analyses performed; comprehensive heterogeneity assessment should be addressed at the author level by improving personal familiarity with appropriate statistical methodology for assessing heterogeneity and involving biostatisticians and epidemiologists in study design, as well as at the editorial level, by mandating adherence to methodologic standards in primary DTA studies and DTA meta-analyses.

Systematic review of adherence to the standards for reporting of diagnostic accuracy studies (STARD) 2015 reporting guideline in cerebral aneurysm imaging diagnostic accuracy studies

BACKGROUND

Diagnostic neuroimaging plays an essential role in guiding clinical decision-making in the management of patients with cerebral aneurysms. Imaging technologies for investigating cerebral aneurysms constantly evolve, and clinicians rely on the published literature to remain up to date. Reporting guidelines have been developed to standardise and strengthen the reporting of clinical evidence. Therefore, it is essential that radiological diagnostic accuracy studies adhere to such guidelines to ensure completeness of reporting. Incomplete reporting hampers the reader's ability to detect bias, determine generalisability of study results or replicate investigation parameters, detracting from the credibility and reliability of studies.

OBJECTIVE

The purpose of this systematic review was to evaluate adherence to the Standards for Reporting of Diagnostic Accuracy Studies (STARD) 2015 reporting guideline amongst imaging diagnostic accuracy studies for cerebral aneurysms.

METHODS

A systematic search for cerebral aneurysm imaging diagnostic accuracy studies was conducted. Journals were cross examined against the STARD 2015 checklist and their compliance with item numbers was recorded.

RESULTS

The search yielded 66 articles. The mean number of STARD items reported was 24.2 ± 2.7 (71.2% ± 7.9%), with a range of 19 to 30 out of a maximum number of 34 items.

CONCLUSION

Taken together, these results indicate that adherence to the STARD 2015 guideline in cerebral aneurysm imaging diagnostic accuracy studies was moderate. Measures to improve compliance include mandating STARD 2015 adherence in instructions to authors issued by journals.

A decade of ubiquicidin development for PET imaging of infection: A systematic review

BACKGROUND

Ubiquicidin is a peptide fragment with selective binding to negatively charged bacterial cell membranes. Besides its earlier labelling with gamma emitting radionuclides, it has been labelled with Positron Emission Tomography (PET) radionuclides in the last decade for imaging infection and distinguishing infectious disease from sterile inflammation. This systematic review aims to evaluate the technology readiness level of PET based ubiquicidin radiopharmaceuticals.

METHODS

Two independent researchers reviewed all articles and abstracts pertaining ubiquicidin and PET imaging that are currently available. Scopus, Google Scholar and PubMed/Medline were used in the search. Upon completion of the literature search all articles and abstracts were evaluated and duplicates were excluded. All non-PET articles as well as review articles without new data were deemed ineligible.

RESULTS

From a total of 17 papers and 10 abstracts the studies were grouped into development, preclinical and clinical studies. Development was published in 15/17 (88%) publications and 6/10 (60%) abstracts, preclinical applications in 9/17 (53%) publications and 1/10 (10%) of abstracts. Finally, clinical studies made up 6/17 (35%) of full publications and 4/10 (40%) of the available abstracts. Development results were the most abundant. All the findings in the different areas of development of ubiquicidin as PET radiopharmaceutical are summarized in this paper.

CONCLUSION

Labelling procedures are generally uncomplicated and relatively fast and there are indications of adequate product stability. The production of PET radiopharmaceuticals based on UBI will therefore not be a barrier for clinical introduction of this technology. Systematization and unification of criteria for preclinical imaging and larger clinical trials are needed to ensure the translation of this radiopharmaceutical into the clinic. Therefore a conclusion with regards to the clinical relevance of ubiquicidin based PET is not yet possible.