Radiologists in the loop: the roles of radiologists in the development of AI applications

Multicentric clinical evaluation of a computed tomography-based fully automated deep neural network for aortic maximum diameter and volumetric measurements

OBJECTIVE

This study aims to evaluate a fully automatic deep learning-based method (augmented radiology for vascular aneurysm [ARVA]) for aortic segmentation and simultaneous diameter and volume measurements.

METHODS

A clinical validation dataset was constructed from preoperative and postoperative aortic computed tomography angiography (CTA) scans for assessing these functions. The dataset totaled 350 computed tomography angiography scans from 216 patients treated at two different hospitals. ARVA's ability to segment the aorta into seven morphologically based aortic segments and measure maximum outer-to-outer wall transverse diameters and compute volumes for each was compared with the measurements of six experts (ground truth) and thirteen clinicians.

RESULTS

Ground truth (experts') measurements of diameters and volumes were manually performed for all aortic segments. The median absolute diameter difference between ground truth and ARVA was 1.6 mm (95% confidence interval [CI], 1.5-1.7; and 1.6 mm [95% CI, 1.6-1.7]) between ground truth and clinicians. ARVA produced measurements within the clinical acceptable range with a proportion of 85.5% (95% CI, 83.5-86.3) compared with the clinicians' 86.0% (95% CI, 83.9-86.0). The median volume similarity error ranged from 0.93 to 0.95 in the main trunk and achieved 0.88 in the iliac arteries.

CONCLUSIONS

This study demonstrates the reliability of a fully automated artificial intelligence-driven solution capable of quick aortic segmentation and analysis of both diameter and volume for each segment.

The endorsement of general and artificial intelligence reporting guidelines in radiological journals: a meta-research study

BACKGROUND

Complete reporting is essential for clinical research. However, the endorsement of reporting guidelines in radiological journals is still unclear. Further, as a field extensively utilizing artificial intelligence (AI), the adoption of both general and AI reporting guidelines would be necessary for enhancing quality and transparency of radiological research. This study aims to investigate the endorsement of general reporting guidelines and those for AI applications in medical imaging in radiological journals, and explore associated journal characteristic variables.

METHODS

This meta-research study screened journals from the Radiology, Nuclear Medicine & Medical Imaging category, Science Citation Index Expanded of the 2022 Journal Citation Reports, and excluded journals not publishing original research, in non-English languages, and instructions for authors unavailable. The endorsement of fifteen general reporting guidelines and ten AI reporting guidelines was rated using a five-level tool: "active strong", "active weak", "passive moderate", "passive weak", and "none". The association between endorsement and journal characteristic variables was evaluated by logistic regression analysis.

RESULTS

We included 117 journals. The top-five endorsed reporting guidelines were CONSORT (Consolidated Standards of Reporting Trials, 58.1%, 68/117), PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses, 54.7%, 64/117), STROBE (STrengthening the Reporting of Observational Studies in Epidemiology, 51.3%, 60/117), STARD (Standards for Reporting of Diagnostic Accuracy, 50.4%, 59/117), and ARRIVE (Animal Research Reporting of In Vivo Experiments, 35.9%, 42/117). The most implemented AI reporting guideline was CLAIM (Checklist for Artificial Intelligence in Medical Imaging, 1.7%, 2/117), while other nine AI reporting guidelines were not mentioned. The Journal Impact Factor quartile and publisher were associated with endorsement of reporting guidelines in radiological journals.

CONCLUSIONS

The general reporting guideline endorsement was suboptimal in radiological journals. The implementation of reporting guidelines for AI applications in medical imaging was extremely low. Their adoption should be strengthened to facilitate quality and transparency of radiological study reporting.

The Development and Validation of an AI Diagnostic Model for Sacroiliitis: A Deep-Learning Approach

PURPOSE

Sacroiliitis refers to the inflammatory condition of the sacroiliac joints, frequently causing lower back pain. It is often associated with systemic conditions. However, its signs on radiographic images can be subtle, which may result in it being overlooked or underdiagnosed. This study aims to utilize artificial intelligence (AI) to create a diagnostic tool for more accurate sacroiliitis detection in radiological images, with the goal of optimizing treatment plans and improving patient outcomes.

MATERIALS AND METHOD

The study included 492 patients who visited our hospital. Right sacroiliac joint films were independently evaluated by two musculoskeletal radiologists using the Modified New York criteria (Normal, Grades 1-4). A consensus reading resolved disagreements. The images were preprocessed with Z-score standardization and histogram equalization. The DenseNet121 algorithm, a convolutional neural network with 201 layers, was used for learning and classification. All steps were performed on the DEEP:PHI platform.

RESULT

The AI model exhibited high accuracy across different grades: 94.53% (Grade 1), 95.83% (Grade 2), 98.44% (Grade 3), 96.88% (Grade 4), and 96.09% (Normal cases). Sensitivity peaked at Grade 3 and Normal cases (100%), while Grade 4 achieved perfect specificity (100%). PPVs ranged from 82.61% (Grade 1) to 100% (Grade 4), and NPVs peaked at 100% for Grade 3 and Normal cases. The F1 scores ranged from 64.41% (Grade 1) to 95.38% (Grade 3).

CONCLUSIONS

The AI diagnostic model showcased a robust performance in detecting and grading sacroiliitis, reflecting its potential to enhance diagnostic accuracy in clinical settings. By facilitating earlier and more accurate diagnoses, this model could substantially impact treatment strategies and patient outcomes.

(De)troubling transparency: artificial intelligence (AI) for clinical applications

Artificial intelligence (AI) and machine learning (ML) techniques occupy a prominent role in medical research in terms of the innovation and development of new technologies. However, while many perceive AI as a technology of promise and hope-one that is allowing for more early and accurate diagnosis-the acceptance of AI and ML technologies in hospitals remains low. A major reason for this is the lack of transparency associated with these technologies, in particular epistemic transparency, which results in AI disturbing or troubling established knowledge practices in clinical contexts. In this article, we describe the development process of one AI application for a clinical setting. We show how epistemic transparency is negotiated and co-produced in close collaboration between AI developers and clinicians and biomedical scientists, forming the context in which AI is accepted as an epistemic operator. Drawing on qualitative research with collaborative researchers developing an AI technology for the early diagnosis of a rare respiratory disease (pulmonary hypertension/PH), this paper examines how including clinicians and clinical scientists in the collaborative practices of AI developers de-troubles transparency. Our research shows how de-troubling transparency occurs in three dimensions of AI development relating to PH: querying of data sets, building software and training the model The close collaboration results in an AI application that is at once social and technological: it integrates and inscribes into the technology the knowledge processes of the different participants in its development. We suggest that it is a misnomer to call these applications 'artificial' intelligence, and that they would be better developed and implemented if they were reframed as forms of sociotechnical intelligence.

Active Learning Performance in Labeling Radiology Images Is 90% Effective

To train artificial intelligence (AI) systems on radiology images, an image labeling step is necessary. Labeling for radiology images usually involves a human radiologist manually drawing a (polygonal) shape onto the image and attaching a word to it. As datasets are typically large, this task is repetitive, time-consuming, error-prone, and expensive. The AI methodology of active learning (AL) can assist human labelers by continuously sorting the unlabeled images in order of information gain and thus getting the labeler always to label the most informative image next. We find that after about 10%, depending on the dataset, of the images in a realistic dataset are labeled, virtually all the information content has been learnt and the remaining images can be automatically labeled. These images can then be checked by the radiologist, which is far easier and faster to do. In this way, the entire dataset is labeled with much less human effort. We introduce AL in detail and expose the effectiveness using three real-life datasets. We contribute five distinct elements to the standard AL workflow creating an advanced methodology.