Preparing Radiologists to Lead in the Era of Artificial Intelligence: Designing and Implementing a Focused Data Science Pathway for Senior Radiology Residents

A scoping review of artificial intelligence in medical education: BEME Guide No. 84

BACKGROUND

Artificial Intelligence (AI) is rapidly transforming healthcare, and there is a critical need for a nuanced understanding of how AI is reshaping teaching, learning, and educational practice in medical education. This review aimed to map the literature regarding AI applications in medical education, core areas of findings, potential candidates for formal systematic review and gaps for future research.

METHODS

This rapid scoping review, conducted over 16 weeks, employed Arksey and O'Malley's framework and adhered to STORIES and BEME guidelines. A systematic and comprehensive search across PubMed/MEDLINE, EMBASE, and MedEdPublish was conducted without date or language restrictions. Publications included in the review spanned undergraduate, graduate, and continuing medical education, encompassing both original studies and perspective pieces. Data were charted by multiple author pairs and synthesized into various thematic maps and charts, ensuring a broad and detailed representation of the current landscape.

RESULTS

The review synthesized 278 publications, with a majority (68%) from North American and European regions. The studies covered diverse AI applications in medical education, such as AI for admissions, teaching, assessment, and clinical reasoning. The review highlighted AI's varied roles, from augmenting traditional educational methods to introducing innovative practices, and underscores the urgent need for ethical guidelines in AI's application in medical education.

CONCLUSION

The current literature has been charted. The findings underscore the need for ongoing research to explore uncharted areas and address potential risks associated with AI use in medical education. This work serves as a foundational resource for educators, policymakers, and researchers in navigating AI's evolving role in medical education. A framework to support future high utility reporting is proposed, the FACETS framework.

A framework to integrate artificial intelligence training into radiology residency programs: preparing the future radiologist

OBJECTIVES

To present a framework to develop and implement a fast-track artificial intelligence (AI) curriculum into an existing radiology residency program, with the potential to prepare a new generation of AI conscious radiologists.

METHODS

The AI-curriculum framework comprises five sequential steps: (1) forming a team of AI experts, (2) assessing the residents' knowledge level and needs, (3) defining learning objectives, (4) matching these objectives with effective teaching strategies, and finally (5) implementing and evaluating the pilot. Following these steps, a multidisciplinary team of AI engineers, radiologists, and radiology residents designed a 3-day program, including didactic lectures, hands-on laboratory sessions, and group discussions with experts to enhance AI understanding. Pre- and post-curriculum surveys were conducted to assess participants' expectations and progress and were analyzed using a Wilcoxon rank-sum test.

RESULTS

There was 100% response rate to the pre- and post-curriculum survey (17 and 12 respondents, respectively). Participants' confidence in their knowledge and understanding of AI in radiology significantly increased after completing the program (pre-curriculum means 3.25 ± 1.48 (SD), post-curriculum means 6.5 ± 0.90 (SD), p-value = 0.002). A total of 75% confirmed that the course addressed topics that were applicable to their work in radiology. Lectures on the fundamentals of AI and group discussions with experts were deemed most useful.

CONCLUSION

Designing an AI curriculum for radiology residents and implementing it into a radiology residency program is feasible using the framework presented. The 3-day AI curriculum effectively increased participants' perception of knowledge and skills about AI in radiology and can serve as a starting point for further customization.

CRITICAL RELEVANCE STATEMENT

The framework provides guidance for developing and implementing an AI curriculum in radiology residency programs, educating residents on the application of AI in radiology and ultimately contributing to future high-quality, safe, and effective patient care.

KEY POINTS

• AI education is necessary to prepare a new generation of AI-conscious radiologists. • The AI curriculum increased participants' perception of AI knowledge and skills in radiology. • This five-step framework can assist integrating AI education into radiology residency programs.

Building Diversity, Equity, and Inclusion Within Radiology Artificial Intelligence: Representation Matters, From Data to the Workforce

Diversity, equity, and inclusion (DEI) is both a critical ingredient and moral imperative in shaping the future of radiology artificial intelligence (AI) for improved patient care, from design to deployment. At the design level: Potential biases and discrimination within data sets results in inaccurate radiology AI models, and there is an urgent need to purposefully embed DEI principles throughout the AI development and implementation process. At the deployment level: Diverse representation in radiology AI leadership, research, and career development is necessary to avoid worsening structural and historical health inequities. To create an inclusive and equitable AI-enabled future in healthcare, a DEI radiology AI leadership training program may be needed to cultivate a diverse and sustainable pipeline of leaders in the field.

Artificial intelligence tools in clinical neuroradiology: essential medico-legal aspects

Commercial software based on artificial intelligence (AI) is entering clinical practice in neuroradiology. Consequently, medico-legal aspects of using Software as a Medical Device (SaMD) become increasingly important. These medico-legal issues warrant an interdisciplinary approach and may affect the way we work in daily practice. In this article, we seek to address three major topics: medical malpractice liability, regulation of AI-based medical devices, and privacy protection in shared medical imaging data, thereby focusing on the legal frameworks of the European Union and the USA. As many of the presented concepts are very complex and, in part, remain yet unsolved, this article is not meant to be comprehensive but rather thought-provoking. The goal is to engage clinical neuroradiologists in the debate and equip them to actively shape these topics in the future.

Artificial Intelligence for Perioperative Medicine: Perioperative Intelligence

The anesthesiologist's role has expanded beyond the operating room, and anesthesiologist-led care teams can deliver coordinated care that spans the entire surgical experience, from preoperative optimization to long-term recovery of surgical patients. This expanded role can help reduce postoperative morbidity and mortality, which are regrettably common, unlike rare intraoperative mortality. Postoperative mortality, if considered a disease category, will be the third leading cause of death just after heart disease and cancer. Rapid advances in technologies like artificial intelligence provide an opportunity to build safe perioperative practices. Artificial intelligence helps by analyzing complex data across disparate systems and producing actionable information. Using artificial intelligence technologies, we can critically examine every aspect of perioperative medicine and devise innovative value-based solutions that can potentially improve patient safety and care delivery, while optimizing cost of care. In this narrative review, we discuss specific applications of artificial intelligence that may help advance all aspects of perioperative medicine, including clinical care, education, quality improvement, and research. We also discuss potential limitations of technology and provide our recommendations for successful adoption.

Artificial intelligence education for radiographers, an evaluation of a UK postgraduate educational intervention using participatory action research: a pilot study

BACKGROUND

Artificial intelligence (AI)-enabled applications are increasingly being used in providing healthcare services, such as medical imaging support. Sufficient and appropriate education for medical imaging professionals is required for successful AI adoption. Although, currently, there are AI training programmes for radiologists, formal AI education for radiographers is lacking. Therefore, this study aimed to evaluate and discuss a postgraduate-level module on AI developed in the UK for radiographers.

METHODOLOGY

A participatory action research methodology was applied, with participants recruited from the first cohort of students enrolled in this module and faculty members. Data were collected using online, semi-structured, individual interviews and focus group discussions. Textual data were processed using data-driven thematic analysis.

RESULTS

Seven students and six faculty members participated in this evaluation. Results can be summarised in the following four themes: a. participants' professional and educational backgrounds influenced their experiences, b. participants found the learning experience meaningful concerning module design, organisation, and pedagogical approaches, c. some module design and delivery aspects were identified as barriers to learning, and d. participants suggested how the ideal AI course could look like based on their experiences.

CONCLUSIONS

The findings of our work show that an AI module can assist educators/academics in developing similar AI education provisions for radiographers and other medical imaging and radiation sciences professionals. A blended learning delivery format, combined with customisable and contextualised content, using an interprofessional faculty approach is recommended for future similar courses.

Artificial Intelligence and Radiology Education

Implementation of artificial intelligence (AI) applications into clinical practice requires AI-savvy radiologists to ensure the safe, ethical, and effective use of these systems for patient care. Increasing demand for AI education reflects recognition of the translation of AI applications from research to clinical practice, with positive trainee attitudes regarding the influence of AI on radiology. However, barriers to AI education, such as limited access to resources, predispose to insufficient preparation for the effective use of AI in practice. In response, national organizations have sponsored formal and self-directed learning courses to provide introductory content on imaging informatics and AI. Foundational courses, such as the National Imaging Informatics Course - Radiology and the Radiological Society of North America Imaging AI Certificate, lay a framework for trainees to explore the creation, deployment, and critical evaluation of AI applications. This report includes additional resources for formal programming courses, video series from leading organizations, and blogs from AI and informatics communities. Furthermore, the scope of "AI and radiology education" includes AI-augmented radiology education, with emphasis on the potential for "precision education" that creates personalized experiences for trainees by accounting for varying learning styles and inconsistent, possibly deficient, clinical case volume. © RSNA, 2022 Keywords: Use of AI in Education, Impact of AI on Education, Artificial Intelligence, Medical Education, Imaging Informatics, Natural Language Processing, Precision Education.

Systematic Review of Radiologist and Medical Student Attitudes on the Role and Impact of AI in Radiology

RATIONALE AND OBJECTIVES

The introduction of AI in radiology has prompted both excitement and hesitation within the field. We performed a systematic review of original studies evaluating the attitudes of radiologists, radiology trainees, and medical students towards AI in radiology.

MATERIALS AND METHODS

We searched PubMed for studies published as of August 24, 2021 for original studies evaluating attitudes of radiologists (attendings and trainees) and medical students towards AI in radiology. We summarized the baseline article characteristics and performed thematic analysis of the questions asked in each study.

RESULTS

Nineteen studies were included evaluating attitudes across different levels of training (medical students, radiology trainees, and radiology attendings) with representation from nearly every continent. Medical students and radiologists alike favored increased educational initiatives, and displayed interest in learning about and implementing AI solutions themselves, despite reporting of a current gap in formal AI training. There was general optimism about the role of AI in radiology, although radiologists and trainees had greater consensus than medical students.

CONCLUSION

Although there is interest in incorporating AI into medical education and optimism among radiologists towards AI, medical students are more divided in their views. We propose that outreach to and AI education for medical students may help improve their attitudes towards the potentially transformative technology of AI for radiology.

Perceptions of US Medical Students on Artificial Intelligence in Medicine: Mixed Methods Survey Study

BACKGROUND

Given the rapidity with which artificial intelligence is gaining momentum in clinical medicine, current physician leaders have called for more incorporation of artificial intelligence topics into undergraduate medical education. This is to prepare future physicians to better work together with artificial intelligence technology. However, the first step in curriculum development is to survey the needs of end users. There has not been a study to determine which media and which topics are most preferred by US medical students to learn about the topic of artificial intelligence in medicine.

OBJECTIVE

We aimed to survey US medical students on the need to incorporate artificial intelligence in undergraduate medical education and their preferred means to do so to assist with future education initiatives.

METHODS

A mixed methods survey comprising both specific questions and a write-in response section was sent through Qualtrics to US medical students in May 2021. Likert scale questions were used to first assess various perceptions of artificial intelligence in medicine. Specific questions were posed regarding learning format and topics in artificial intelligence.

RESULTS

We surveyed 390 US medical students with an average age of 26 (SD 3) years from 17 different medical programs (the estimated response rate was 3.5%). A majority (355/388, 91.5%) of respondents agreed that training in artificial intelligence concepts during medical school would be useful for their future. While 79.4% (308/388) were excited to use artificial intelligence technologies, 91.2% (353/387) either reported that their medical schools did not offer resources or were unsure if they did so. Short lectures (264/378, 69.8%), formal electives (180/378, 47.6%), and Q and A panels (167/378, 44.2%) were identified as preferred formats, while fundamental concepts of artificial intelligence (247/379, 65.2%), when to use artificial intelligence in medicine (227/379, 59.9%), and pros and cons of using artificial intelligence (224/379, 59.1%) were the most preferred topics for enhancing their training.

CONCLUSIONS

The results of this study indicate that current US medical students recognize the importance of artificial intelligence in medicine and acknowledge that current formal education and resources to study artificial intelligence-related topics are limited in most US medical schools. Respondents also indicated that a hybrid formal/flexible format would be most appropriate for incorporating artificial intelligence as a topic in US medical schools. Based on these data, we conclude that there is a definitive knowledge gap in artificial intelligence education within current medical education in the US. Further, the results suggest there is a disparity in opinions on the specific format and topics to be introduced.

Data governance functions to support responsible data stewardship in pediatric radiology research studies using artificial intelligence

The integration of human and machine intelligence promises to profoundly change the practice of medicine. The rapidly increasing adoption of artificial intelligence (AI) solutions highlights its potential to streamline physician work and optimize clinical decision-making, also in the field of pediatric radiology. Large imaging databases are necessary for training, validating and testing these algorithms. To better promote data accessibility in multi-institutional AI-enabled radiologic research, these databases centralize the large volumes of data required to effect accurate models and outcome predictions. However, such undertakings must consider the sensitivity of patient information and therefore utilize requisite data governance measures to safeguard data privacy and security, to recognize and mitigate the effects of bias and to promote ethical use. In this article we define data stewardship and data governance, review their key considerations and applicability to radiologic research in the pediatric context, and consider the associated best practices along with the ramifications of poorly executed data governance. We summarize several adaptable data governance frameworks and describe strategies for their implementation in the form of distributed and centralized approaches to data management.

Code and Data Sharing Practices in the Radiology Artificial Intelligence Literature: A Meta-Research Study

PURPOSE

To evaluate code and data sharing practices in original artificial intelligence (AI) scientific manuscripts published in the Radiological Society of North America (RSNA) journals suite from 2017 through 2021.

MATERIALS AND METHODS

A retrospective meta-research study was conducted of articles published in the RSNA journals suite from January 1, 2017, through December 31, 2021. A total of 218 articles were included and evaluated for code sharing practices, reproducibility of shared code, and data sharing practices. Categorical comparisons were conducted using Fisher exact tests with respect to year and journal of publication, author affiliation(s), and type of algorithm used.

RESULTS

Of the 218 included articles, 73 (34%) shared code, with 24 (33% of code sharing articles and 11% of all articles) sharing reproducible code. Radiology and Radiology: Artificial Intelligence published the most code sharing articles (48 [66%] and 21 [29%], respectively). Twenty-nine articles (13%) shared data, and 12 of these articles (41% of data sharing articles) shared complete experimental data by using only public domain datasets. Four of the 218 articles (2%) shared both code and complete experimental data. Code sharing rates were statistically higher in 2020 and 2021 compared with earlier years (P < .01) and were higher in Radiology and Radiology: Artificial Intelligence compared with other journals (P < .01).

CONCLUSION

Original AI scientific articles in the RSNA journals suite had low rates of code and data sharing, emphasizing the need for open-source code and data to achieve transparent and reproducible science.Keywords: Meta-Analysis, AI in Education, Machine LearningSupplemental material is available for this article.© RSNA, 2022.

Diagnostic captioning: a survey

Diagnostic captioning (DC) concerns the automatic generation of a diagnostic text from a set of medical images of a patient collected during an examination. DC can assist inexperienced physicians, reducing clinical errors. It can also help experienced physicians produce diagnostic reports faster. Following the advances of deep learning, especially in generic image captioning, DC has recently attracted more attention, leading to several systems and datasets. This article is an extensive overview of DC. It presents relevant datasets, evaluation measures, and up-to-date systems. It also highlights shortcomings that hinder DC’s progress and proposes future directions.

The current state of knowledge on imaging informatics: a survey among Spanish radiologists

Background

There is growing concern about the impact of artificial intelligence (AI) on radiology and the future of the profession. The aim of this study is to evaluate general knowledge and concerns about trends on imaging informatics among radiologists working in Spain (residents and attending physicians). For this purpose, an online survey among radiologists working in Spain was conducted with questions related to: knowledge about terminology and technologies, need for a regulated academic training on AI and concerns about the implications of the use of these technologies.

Results

A total of 223 radiologists answered the survey, of whom 76.7% were attending physicians and 23.3% residents. General terms such as AI and algorithm had been heard of or read in at least 75.8% and 57.4% of the cases, respectively, while more specific terms were scarcely known. All the respondents consider that they should pursue academic training in medical informatics and new technologies, and 92.9% of them reckon this preparation should be incorporated in the training program of the specialty. Patient safety was found to be the main concern for 54.2% of the respondents. Job loss was not seen as a peril by 45.7% of the participants.

Conclusions

Although there is a lack of knowledge about AI among Spanish radiologists, there is a will to explore such topics and a general belief that radiologists should be trained in these matters. Based on the results, a consensus is needed to change the current training curriculum to better prepare future radiologists.

Multi-Stage Harmonization for Robust AI across Breast MR Databases

Simple Summary

Batch harmonization of radiomic features extracted from magnetic resonance images of breast lesions from two databases was applied to an artificial intelligence/machine learning classification workflow. Training and independent test sets from the two databases, as well as the combination of them, were used in pre-harmonization and post-harmonization forms to investigate the generalizability of performance in the task of distinguishing between malignant and benign lesions. Most training and independent test scenarios were statistically equivalent, demonstrating that batch harmonization with feature selection harmonization can potentially develop generalizable classification models.

Abstract

Radiomic features extracted from medical images may demonstrate a batch effect when cases come from different sources. We investigated classification performance using training and independent test sets drawn from two sources using both pre-harmonization and post-harmonization features. In this retrospective study, a database of thirty-two radiomic features, extracted from DCE-MR images of breast lesions after fuzzy c-means segmentation, was collected. There were 944 unique lesions in Database A (208 benign lesions, 736 cancers) and 1986 unique lesions in Database B (481 benign lesions, 1505 cancers). The lesions from each database were divided by year of image acquisition into training and independent test sets, separately by database and in combination. ComBat batch harmonization was conducted on the combined training set to minimize the batch effect on eligible features by database. The empirical Bayes estimates from the feature harmonization were applied to the eligible features of the combined independent test set. The training sets (A, B, and combined) were then used in training linear discriminant analysis classifiers after stepwise feature selection. The classifiers were then run on the A, B, and combined independent test sets. Classification performance was compared using pre-harmonization features to post-harmonization features, including their corresponding feature selection, evaluated using the area under the receiver operating characteristic curve (AUC) as the figure of merit. Four out of five training and independent test scenarios demonstrated statistically equivalent classification performance when compared pre- and post-harmonization. These results demonstrate that translation of machine learning techniques with batch data harmonization can potentially yield generalizable models that maintain classification performance.

A Conference-Friendly, Hands-on Introduction to Deep Learning for Radiology Trainees

Artificial or augmented intelligence, machine learning, and deep learning will be an increasingly important part of clinical practice for the next generation of radiologists. It is therefore critical that radiology residents develop a practical understanding of deep learning in medical imaging. Certain aspects of deep learning are not intuitive and may be better understood through hands-on experience; however, the technical requirements for setting up a programming and computing environment for deep learning can pose a high barrier to entry for individuals with limited experience in computer programming and limited access to GPU-accelerated computing. To address these concerns, we implemented an introductory module for deep learning in medical imaging within a self-contained, web-hosted development environment. Our initial experience established the feasibility of guiding radiology trainees through the module within a 45-min period typical of educational conferences.

Natural Language Processing of Radiology Text Reports: Interactive Text Classification

This report presents a hands-on introduction to natural language processing (NLP) of radiology reports with deep neural networks in Google Colaboratory (Colab) to introduce readers to the rapidly evolving field of NLP. The implementation of the Google Colab notebook was designed with code hidden to facilitate learning for noncoders (ie, individuals with little or no computer programming experience). The data used for this module are the corpus of radiology reports from the Indiana University chest x-ray collection available from the National Library of Medicine's Open-I service. The module guides learners through the process of exploring the data, splitting the data for model training and testing, preparing the data for NLP analysis, and training a deep NLP model to classify the reports as normal or abnormal. Concepts in NLP, such as tokenization, numericalization, language modeling, and word embeddings, are demonstrated in the module. The module is implemented in a guided fashion with the authors presenting the material and explaining concepts. Interactive features and extensive text commentary are provided directly in the notebook to facilitate self-guided learning and experimentation with the module. Keywords: Neural Networks, Negative Expression Recognition, Natural Language Processing, Computer Applications, Informatics © RSNA, 2021.

Radiologists in the loop: the roles of radiologists in the development of AI applications

Objectives

To examine the various roles of radiologists in different steps of developing artificial intelligence (AI) applications.

Materials and methods

Through the case study of eight companies active in developing AI applications for radiology, in different regions (Europe, Asia, and North America), we conducted 17 semi-structured interviews and collected data from documents. Based on systematic thematic analysis, we identified various roles of radiologists. We describe how each role happens across the companies and what factors impact how and when these roles emerge.

Results

We identified 9 roles that radiologists play in different steps of developing AI applications: (1) problem finder (in 4 companies); (2) problem shaper (in 3 companies); (3) problem dominator (in 1 company); (4) data researcher (in 2 companies); (5) data labeler (in 3 companies); (6) data quality controller (in 2 companies); (7) algorithm shaper (in 3 companies); (8) algorithm tester (in 6 companies); and (9) AI researcher (in 1 company).

Conclusions

Radiologists can play a wide range of roles in the development of AI applications. How actively they are engaged and the way they are interacting with the development teams significantly vary across the cases. Radiologists need to become proactive in engaging in the development process and embrace new roles.

Key Points

• Radiologists can play a wide range of roles during the development of AI applications.

• Both radiologists and developers need to be open to new roles and ways of interacting during the development process.

• The availability of resources, time, expertise, and trust are key factors that impact how actively radiologists play roles in the development process.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00330-021-07879-w.