Review of Artificial Intelligence Training Tools and Courses for Radiologists

The effect of machine learning algorithms in the prediction, and diagnosis of meningitis: A systematic review

Background and Aims

This systematic review aimed to evaluating the effectiveness of machine learning (ML) algorithms for the prediction and diagnosis of meningitis.

Methods

On November 12, 2022, a systematic review was carried out using a keyword search in the reliable scientific databases PubMed, EMBASE, Scopus, and Web of Science. The recommendations of Preferred Reporting for Systematic Reviews and Meta-Analyses (PRISMA) were adhered to. Studies conducted in English that employed ML to predict and identify meningitis were deemed to match the inclusion criteria. The eligibility requirements were used to independently review the titles and abstracts. The whole text was then obtained and independently reviewed in accordance with the eligibility requirements.

Results

After all the research matched the inclusion criteria, a total of 16 studies were added to the systematic review. Studies on the application of ML algorithms in the three categories of disease diagnosis ability (8.16) and disease prediction ability (8.16) (including cases related to identifying patients (50%), risk of death in patients (25%), the consequences of the disease in childhood (12.5%), and its etiology [12.5%]) were placed. Among the ML algorithms used in this study, logistic regression (LR) (4.16, 25%) and multiple logistic regression (MLR) (4.16, 25%) were the most used. All the included studies indicated improvements in the processes of diagnosis, prediction, and disease outbreak with the help of ML algorithms.

Conclusion

The results of the study showed that in all included studies, ML algorithms were an effective approach to facilitate diagnosis, predict consequences for risk classification, and improve resource utilization by predicting the volume of patients or services as well as discovering risk factors. The role of ML algorithms in improving disease diagnosis was more significant than disease prediction and prevalence. Meanwhile, the use of combined methods can optimize differential diagnoses and facilitate the decision-making process for healthcare providers.

Development and validation of an educational software based in artificial neural networks for training in radiology (JORCAD) through an interactive learning activity

The use of Computer Aided Detection (CAD) software has been previously documented as a valuable tool to improve specialist training in Radiology. This research assesses the utility of an educational software tool aimed to train residents in Radiology and other related medical specialties and students from Medicine degree. This in-house developed software, called JORCAD, integrates a CAD system based in Convolutional Neural Networks (CNNs) with annotated cases from radiological image databases. The methodology followed for software validation was expert judgement after completing an interactive learning activity. Participants received a theoretical session and a software usage tutorial and afterwards utilized the application in a dedicated workstation to analyze a series of proposed cases of thorax computed tomography (CT) and mammography. A total of 26 expert participants from the Radiology Department at Salamanca University Hospital (15 specialists and 11 residents) fulfilled the activity and evaluated different aspects through a series of surveys: software usability, case navigation tools, CAD module utility for learning and JORCAD educational capabilities. Participants also graded imaging cases to establish JORCAD usefulness for training radiology residents. According to the statistical analysis of survey results and expert cases scoring, along with their opinions, it can be concluded that JORCAD software is a useful tool for training future specialists. The combination of CAD with annotated cases from validated databases enhances learning, offering a second opinion and changing the usual training paradigm. Including software as JORCAD in residency training programs of Radiology and other medical specialties would have a positive effect on trainees' background knowledge.

Developing medical imaging AI for emerging infectious diseases

Very few of the COVID-19 ML models were fit for deployment in real-world settings. In this Comment, Huang et al. discuss the main steps required to develop clinically useful models in the context of an emerging infectious disease.

Perspective Toward Machine Learning Implementation in Pediatric Medicine: Mixed Methods Study

Background

Given the costs of machine learning implementation, a systematic approach to prioritizing which models to implement into clinical practice may be valuable.

Objective

The primary objective was to determine the health care attributes respondents at 2 pediatric institutions rate as important when prioritizing machine learning model implementation. The secondary objective was to describe their perspectives on implementation using a qualitative approach.

Methods

In this mixed methods study, we distributed a survey to health system leaders, physicians, and data scientists at 2 pediatric institutions. We asked respondents to rank the following 5 attributes in terms of implementation usefulness: the clinical problem was common, the clinical problem caused substantial morbidity and mortality, risk stratification led to different actions that could reasonably improve patient outcomes, reducing physician workload, and saving money. Important attributes were those ranked as first or second most important. Individual qualitative interviews were conducted with a subsample of respondents.

Results

Among 613 eligible respondents, 275 (44.9%) responded. Qualitative interviews were conducted with 17 respondents. The most common important attributes were risk stratification leading to different actions (205/275, 74.5%) and clinical problem causing substantial morbidity or mortality (177/275, 64.4%). The attributes considered least important were reducing physician workload and saving money. Qualitative interviews consistently prioritized implementations that improved patient outcomes.

Conclusions

Respondents prioritized machine learning model implementation where risk stratification would lead to different actions and clinical problems that caused substantial morbidity and mortality. Implementations that improved patient outcomes were prioritized. These results can help provide a framework for machine learning model implementation.

Quality Assurance of a Cross-Border and Sub-Specialized Teleradiology Service

Background: The current literature discusses aspects of quality assurance (QA) and sub-specialization. However, the challenges of these topics in a teleradiology network have been less explored. In a project report, we aimed to review the development and enforcement of sub-specialized radiology at Telemedicine Clinic (TMC), one of the largest teleradiology providers in Europe, and to describe each step of its QA. Evaluation: The company-specific background was provided by the co-authors—current and former staff members of TMC. Detailed descriptions of the structures of sub-specialization and QA at TMC are provided. Exemplary quantitative evaluation of caseloads and disagreement rates of secondary reviews are illustrated. Description of Sub-specialization and Quality Assurance at TMC: Sub-specialization at TMC is divided into musculoskeletal radiology, neuroradiology, head and neck, a body, and an emergency section operating at local daytime in Europe and Australia. Quality assurance is based on a strict selection process of radiologists, specific reporting guidelines, feedback through the secondary reading of 100% of all radiology reports for new starters, and a minimum of 5% of radiology reports on a continuous basis for all other radiologists, knowledge sharing activities and ongoing training. The level of sub-specialization of each radiologist is monitored continuously on an individual basis in detail. After prospective secondary readings, the mean disagreement rate at TMC indicating at least possibly clinically relevant findings was 4% in 2021. Conclusion: With continuing and current developments in radiology in mind, the essential features of sub-specialization and innovative QA are relevant for further expansion of teleradiology services and for most radiology departments worldwide to respond to the increasing demand for value-based radiology.

A "Bumper-Car" Curriculum for Teaching Deep Learning to Radiology Residents☆

RATIONALE AND OBJECTIVES

Our goal was to create an artificial intelligence (AI) training curriculum for residents that taught them to create, train, evaluate and refine deep learning (DL) models. Hands-on training of models was emphasized and didactic presentations of the mathematical and programmatic underpinnings of DL were minimized.

MATERIALS AND METHODS

We created a three-session, 6-hour curriculum based on a "no-code" machine learning system called Lobe.ai. This class met weekly in June 2021. Pre-class homework included reading assignments, software installation, dataset downloads, and image-collection and labeling. The class sessions included several short, didactic presentations, but were largely devoted to hands-on training of DL models. After the course, our residents completed a short, anonymous, online survey about the course.

RESULTS

Our residents learned to acquire and label a wide variety of image datasets. They quickly learned to train DL models to classify these datasets, as well as how to evaluate and refine these models. Our survey showed that most residents felt AI to be important and worth learning, but most were not very interested in learning to program. Most felt that the course taught them useful things about DL, and they were now more interested in the topic. Most would recommend the course to other residents, as well as to medical students and to radiology faculty.

CONCLUSION

The course met our objectives of teaching our residents to create, train, evaluate, and refine DL models. We hope that the hands-on experience they gained in this course will enable them to recognize problems in diagnostic AI systems, and to help solve such problems in their own radiology practices.