Clinical use of artificial intelligence products for radiology in the Netherlands between 2020 and 2022

Patient perspectives on the use of artificial intelligence in prostate cancer diagnosis on MRI

OBJECTIVES

This study investigated patients' acceptance of artificial intelligence (AI) for diagnosing prostate cancer (PCa) on MRI scans and the factors influencing their trust in AI diagnoses.

MATERIALS AND METHODS

A prospective, multicenter study was conducted between January and November 2023. Patients undergoing prostate MRI were surveyed about their opinions on hypothetical AI assessment of their MRI scans. The questionnaire included nine items: four on hypothetical scenarios of combinations between AI and the radiologist, two on trust in the diagnosis, and three on accountability for misdiagnosis. Relationships between the items and independent variables were assessed using multivariate analysis.

RESULTS

A total of 212 PCa suspicious patients undergoing prostate MRI were included. The majority preferred AI involvement in their PCa diagnosis alongside a radiologist, with 91% agreeing with AI as the primary reader and 79% as the secondary reader. If AI has a high certainty diagnosis, 15% of the respondents would accept it as the sole decision-maker. Autonomous AI outperforming radiologists would be accepted by 52%. Higher educated persons tended to accept AI when it would outperform radiologists (p < 0.05). The respondents indicated that the hospital (76%), radiologist (70%), and program developer (55%) should be held accountable for misdiagnosis.

CONCLUSIONS

Patients favor AI involvement alongside radiologists in PCa diagnosis. Trust in AI diagnosis depends on the patient's education level and the AI performance, with autonomous AI acceptance by a small majority on the condition that AI outperforms a radiologist. Respondents held the hospital, radiologist, and program developers accountable for misdiagnosis in descending order of accountability.

CLINICAL RELEVANCE STATEMENT

Patients show a high level of acceptance for AI-assisted prostate cancer diagnosis on MRI, either alongside radiologists or fully autonomous, particularly if it demonstrates superior performance to radiologists alone.

KEY POINTS

Prostate cancer suspicious patients may accept autonomous AI based on performance. Patients prefer AI involvement alongside a radiologist in diagnosing prostate cancer. Patients indicate accountability for AI should be shared among multiple stakeholders.

Using AI to Identify Unremarkable Chest Radiographs for Automatic Reporting

Background Radiology practices have a high volume of unremarkable chest radiographs and artificial intelligence (AI) could possibly improve workflow by providing an automatic report. Purpose To estimate the proportion of unremarkable chest radiographs, where AI can correctly exclude pathology (ie, specificity) without increasing diagnostic errors. Materials and Methods In this retrospective study, consecutive chest radiographs in unique adult patients (≥18 years of age) were obtained January 1-12, 2020, at four Danish hospitals. Exclusion criteria included insufficient radiology reports or AI output error. Two thoracic radiologists, who were blinded to AI output, labeled chest radiographs as "remarkable" or "unremarkable" based on predefined unremarkable findings (reference standard). Radiology reports were classified similarly. A commercial AI tool was adapted to output a chest radiograph "remarkableness" probability, which was used to calculate specificity at different AI sensitivities. Chest radiographs with missed findings by AI and/or the radiology report were graded by one thoracic radiologist as critical, clinically significant, or clinically insignificant. Paired proportions were compared using the McNemar test. Results A total of 1961 patients were included (median age, 72 years [IQR, 58-81 years]; 993 female), with one chest radiograph per patient. The reference standard labeled 1231 of 1961 chest radiographs (62.8%) as remarkable and 730 of 1961 (37.2%) as unremarkable. At 99.9%, 99.0%, and 98.0% sensitivity, the AI had a specificity of 24.5% (179 of 730 radiographs [95% CI: 21, 28]), 47.1% (344 of 730 radiographs [95% CI: 43, 51]), and 52.7% (385 of 730 radiographs [95% CI: 49, 56]), respectively. With the AI fixed to have a similar sensitivity as radiology reports (87.2%), the missed findings of AI and reports had 2.2% (27 of 1231 radiographs) and 1.1% (14 of 1231 radiographs) classified as critical (P = .01), 4.1% (51 of 1231 radiographs) and 3.6% (44 of 1231 radiographs) classified as clinically significant (P = .46), and 6.5% (80 of 1231) and 8.1% (100 of 1231) classified as clinically insignificant (P = .11), respectively. At sensitivities greater than or equal to 95.4%, the AI tool exhibited less than or equal to 1.1% critical misses. Conclusion A commercial AI tool used off-label could correctly exclude pathology in 24.5%-52.7% of all unremarkable chest radiographs at greater than or equal to 98% sensitivity. The AI had equal or lower rates of critical misses than radiology reports at sensitivities greater than or equal to 95.4%. These results should be confirmed in a prospective study. © RSNA, 2024 Supplemental material is available for this article. See also the editorial by Yoon and Hwang in this issue.

What makes a good scientific presentation on artificial intelligence in medical imaging?

PURPOSE

Adequate communication of scientific findings is crucial to enhance knowledge transfer. This study aimed to determine the key features of a good scientific oral presentation on artificial intelligence (AI) in medical imaging.

METHODS

A total of 26 oral presentations dealing with original research on AI studies in medical imaging at the 2023 RSNA annual meeting were included and systematically assessed by three observers. The presentation quality of the research question, inclusion criteria, reference standard, method, results, clinical impact, presentation clarity, presenter engagement, and the presentation's quality of knowledge transfer were assessed using five-point Likert scales. The number of slides, the average number of words per slide, the number of interactive slides, the number of figures, and the number of tables were also determined for each presentation. Mixed-effects ordinal regression was used to assess the association between the above-mentioned variables and the quality of knowledge transfer of the presentation.

RESULTS

A significant positive association was found between the quality of the presentation of the research question and the presentation's quality of knowledge transfer (odds ratio [OR]: 2.5, P = 0.005). The average number of words per slide was significantly negatively associated with the presentation's quality of knowledge transfer (OR: 0.9, P < 0.001). No other significant associations were found.

CONCLUSION

Researchers who orally present their scientific findings in the field of AI and medical imaging should pay attention to clearly communicating their research question and minimizing the number of words per slide to maximize the value of their presentation.

The American Society of Radiologic Technologists (ASRT) AI educator survey: A cross-sectional study to explore knowledge, experience, and use of AI within education

INTRODUCTION

Artificial Intelligence (AI) is revolutionizing medical imaging and radiation therapy. AI-powered applications are being deployed to aid Medical Radiation Technologists (MRTs) in clinical workflows, decision-making, dose optimisation, and a wide range of other tasks. Exploring the levels of AI education provided across the United States is crucial to prepare future graduates to deliver the digital future. This study aims to assess educators' levels of AI knowledge, the current state of AI educational provisions, the perceived challenges around AI education, and important factors for future advancements.

METHODS

An online survey was electronically administered to all radiologic technologists in the American Society of Radiologic Technologists (ASRT) database who indicated that they had an educator role in the United States. This was distributed through the membership of the ASRT, from February to April 2023. All quantitative data was analysed using frequency and descriptive statistics. The survey's open-ended questions were analysed using a conceptual content analysis approach.

RESULTS

Out of 5,066 educators in the ASRT database, 373 valid responses were received, resulting in a response rate of 7.4%. Despite 84.5% of educators expressing the importance of teaching AI, 23.7% currently included AI in academic curricula. Of the 76.3% that did not include AI in their curricula, lack of AI knowledge among educators was the top reason for not integrating AI in education (59.1%). Similarly, AI-enabled tools were utilised by only 11.1% of the programs to assist teaching. The levels of trust in AI varied among educators.

CONCLUSION

The study found that although US educators of MRTs have a good baseline knowledge of general concepts regarding AI, they could improve on the teaching and use of AI in their curricula. AI training and guidance, adequate time to develop educational resources, and funding and support from higher education institutions were key priorities as highlighted by educators.

External validation of the RSNA 2020 pulmonary embolism detection challenge winning deep learning algorithm

PURPOSE

To evaluate the diagnostic performance and generalizability of the winning DL algorithm of the RSNA 2020 PE detection challenge to a local population using CTPA data from two hospitals.

MATERIALS AND METHODS

Consecutive CTPA images from patients referred for suspected PE were retrospectively analysed. The winning RSNA 2020 DL algorithm was retrained on the RSNA-STR Pulmonary Embolism CT (RSPECT) dataset. The algorithm was tested in hospital A on multidetector CT (MDCT) images of 238 patients and in hospital B on spectral detector CT (SDCT) and virtual monochromatic images (VMI) of 114 patients. The output of the DL algorithm was compared with a reference standard, which included a consensus reading by at least two experienced cardiothoracic radiologists for both hospitals. Areas under the receiver operating characteristic curve (AUCs) were calculated. Sensitivity and specificity were determined using the maximum Youden index.

RESULTS

According to the reference standard, PE was present in 73 patients (30.7%) in hospital A and 33 patients (29.0%) in hospital B. For the DL algorithm the AUC was 0.96 (95% CI 0.92-0.98) in hospital A, 0.89 (95% CI 0.81-0.94) for conventional reconstruction in hospital B and 0.87 (95% CI 0.80-0.93) for VMI.

CONCLUSION

The RSNA 2020 pulmonary embolism detection on CTPA challenge winning DL algorithm, retrained on the RSPECT dataset, showed high diagnostic accuracy on MDCT images. A somewhat lower performance was observed on SDCT images, which suggest additional training on novel CT technology may improve generalizability of this DL algorithm.

A novel reporting workflow for automated integration of artificial intelligence results into structured radiology reports

OBJECTIVES

Artificial intelligence (AI) has tremendous potential to help radiologists in daily clinical routine. However, a seamless, standardized, and time-efficient way of integrating AI into the radiology workflow is often lacking. This constrains the full potential of this technology. To address this, we developed a new reporting pipeline that enables automated pre-population of structured reports with results provided by AI tools.

METHODS

Findings from a commercially available AI tool for chest X-ray pathology detection were sent to an IHE-MRRT-compliant structured reporting (SR) platform as DICOM SR elements and used to automatically pre-populate a chest X-ray SR template. Pre-populated AI results could be validated, altered, or deleted by radiologists accessing the SR template. We assessed the performance of this newly developed AI to SR pipeline by comparing reporting times and subjective report quality to reports created as free-text and conventional structured reports.

RESULTS

Chest X-ray reports with the new pipeline could be created in significantly less time than free-text reports and conventional structured reports (mean reporting times: 66.8 s vs. 85.6 s and 85.8 s, respectively; both p < 0.001). Reports created with the pipeline were rated significantly higher quality on a 5-point Likert scale than free-text reports (p < 0.001).

CONCLUSION

The AI to SR pipeline offers a standardized, time-efficient way to integrate AI-generated findings into the reporting workflow as parts of structured reports and has the potential to improve clinical AI integration and further increase synergy between AI and SR in the future.

CRITICAL RELEVANCE STATEMENT

With the AI-to-structured reporting pipeline, chest X-ray reports can be created in a standardized, time-efficient, and high-quality manner. The pipeline has the potential to improve AI integration into daily clinical routine, which may facilitate utilization of the benefits of AI to the fullest.

KEY POINTS

• A pipeline was developed for automated transfer of AI results into structured reports. • Pipeline chest X-ray reporting is faster than free-text or conventional structured reports. • Report quality was also rated higher for reports created with the pipeline. • The pipeline offers efficient, standardized AI integration into the clinical workflow.