Checklist for Artificial Intelligence in Medical Imaging (CLAIM): A Guide for Authors and Reviewers

Empirical Study on Classifiers for Earlier Prediction of COVID-19 Infection Cure and Death Rate in the Indian States

Machine Learning methods can play a key role in predicting the spread of respiratory infection with the help of predictive analytics. Machine Learning techniques help mine data to better estimate and predict the COVID-19 infection status. A Fine-tuned Ensemble Classification approach for predicting the death and cure rates of patients from infection using Machine Learning techniques has been proposed for different states of India. The proposed classification model is applied to the recent COVID-19 dataset for India, and a performance evaluation of various state-of-the-art classifiers to the proposed model is performed. The classifiers forecasted the patients’ infection status in different regions to better plan resources and response care systems. The appropriate classification of the output class based on the extracted input features is essential to achieve accurate results of classifiers. The experimental outcome exhibits that the proposed Hybrid Model reached a maximum F1-score of 94% compared to Ensembles and other classifiers like Support Vector Machine, Decision Trees, and Gaussian Naïve Bayes on a dataset of 5004 instances through 10-fold cross-validation for predicting the right class. The feasibility of automated prediction for COVID-19 infection cure and death rates in the Indian states was demonstrated.

Machine Learning for Workflow Applications in Screening Mammography: Systematic Review and Meta-Analysis

Background Advances in computer processing and improvements in data availability have led to the development of machine learning (ML) techniques for mammographic imaging. Purpose To evaluate the reported performance of stand-alone ML applications for screening mammography workflow. Materials and Methods Ovid Embase, Ovid Medline, Cochrane Central Register of Controlled Trials, Scopus, and Web of Science literature databases were searched for relevant studies published from January 2012 to September 2020. The study was registered with the PROSPERO International Prospective Register of Systematic Reviews (protocol no. CRD42019156016). Stand-alone technology was defined as a ML algorithm that can be used independently of a human reader. Studies were quality assessed using the Quality Assessment of Diagnostic Accuracy Studies 2 and the Prediction Model Risk of Bias Assessment Tool, and reporting was evaluated using the Checklist for Artificial Intelligence in Medical Imaging. A primary meta-analysis included the top-performing algorithm and corresponding reader performance from which pooled summary estimates for the area under the receiver operating characteristic curve (AUC) were calculated using a bivariate model. Results Fourteen articles were included, which detailed 15 studies for stand-alone detection (n = 8) and triage (n = 7). Triage studies reported that 17%-91% of normal mammograms identified could be read by adapted screening, while "missing" an estimated 0%-7% of cancers. In total, an estimated 185 252 cases from three countries with more than 39 readers were included in the primary meta-analysis. The pooled sensitivity, specificity, and AUC was 75.4% (95% CI: 65.6, 83.2; P = .11), 90.6% (95% CI: 82.9, 95.0; P = .40), and 0.89 (95% CI: 0.84, 0.98), respectively, for algorithms, and 73.0% (95% CI: 60.7, 82.6), 88.6% (95% CI: 72.4, 95.8), and 0.85 (95% CI: 0.78, 0.97), respectively, for readers. Conclusion Machine learning (ML) algorithms that demonstrate a stand-alone application in mammographic screening workflows achieve or even exceed human reader detection performance and improve efficiency. However, this evidence is from a small number of retrospective studies. Therefore, further rigorous independent external prospective testing of ML algorithms to assess performance at preassigned thresholds is required to support these claims. ©RSNA, 2021 Online supplemental material is available for this article. See also the editorial by Whitman and Moseley in this issue.

Checklist for Evaluation of Image-Based Artificial Intelligence Reports in Dermatology: CLEAR Derm Consensus Guidelines From the International Skin Imaging Collaboration Artificial Intelligence Working Group

IMPORTANCE

The use of artificial intelligence (AI) is accelerating in all aspects of medicine and has the potential to transform clinical care and dermatology workflows. However, to develop image-based algorithms for dermatology applications, comprehensive criteria establishing development and performance evaluation standards are required to ensure product fairness, reliability, and safety.

OBJECTIVE

To consolidate limited existing literature with expert opinion to guide developers and reviewers of dermatology AI.

EVIDENCE REVIEW

In this consensus statement, the 19 members of the International Skin Imaging Collaboration AI working group volunteered to provide a consensus statement. A systematic PubMed search was performed of English-language articles published between December 1, 2008, and August 24, 2021, for "artificial intelligence" and "reporting guidelines," as well as other pertinent studies identified by the expert panel. Factors that were viewed as critical to AI development and performance evaluation were included and underwent 2 rounds of electronic discussion to achieve consensus.

FINDINGS

A checklist of items was developed that outlines best practices of image-based AI development and assessment in dermatology.

CONCLUSIONS AND RELEVANCE

Clinically effective AI needs to be fair, reliable, and safe; this checklist of best practices will help both developers and reviewers achieve this goal.

Identifying clinically applicable machine learning algorithms for glioma segmentation: recent advances and discoveries

Background

While there are innumerable machine learning (ML) research algorithms used for segmentation of gliomas, there is yet to be a US FDA cleared product. The aim of this study is to explore the systemic limitations of research algorithms that have prevented translation from concept to product by a review of the current research literature.

Methods

We performed a systematic literature review on 4 databases. Of 11 727 articles, 58 articles met the inclusion criteria and were used for data extraction and screening using TRIPOD.

Results

We found that while many articles were published on ML-based glioma segmentation and report high accuracy results, there were substantial limitations in the methods and results portions of the papers that result in difficulty reproducing the methods and translation into clinical practice.

Conclusions

In addition, we identified that more than a third of the articles used the same publicly available BRaTS and TCIA datasets and are responsible for the majority of patient data on which ML algorithms were trained, which leads to limited generalizability and potential for overfitting and bias.

Model for ASsessing the value of Artificial Intelligence in medical imaging (MAS-AI)

Objectives

Artificial intelligence (AI) is seen as a major disrupting force in the future healthcare system. However, the assessment of the value of AI technologies is still unclear. Therefore, a multidisciplinary group of experts and patients developed a Model for ASsessing the value of AI (MAS-AI) in medical imaging. Medical imaging is chosen due to the maturity of AI in this area, ensuring a robust evidence-based model.

Methods

MAS-AI was developed in three phases. First, a literature review of existing guides, evaluations, and assessments of the value of AI in the field of medical imaging. Next, we interviewed leading researchers in AI in Denmark. The third phase consisted of two workshops where decision makers, patient organizations, and researchers discussed crucial topics for evaluating AI. The multidisciplinary team revised the model between workshops according to comments.

Results

The MAS-AI guideline consists of two steps covering nine domains and five process factors supporting the assessment. Step 1 contains a description of patients, how the AI model was developed, and initial ethical and legal considerations. In step 2, a multidisciplinary assessment of outcomes of the AI application is done for the five remaining domains: safety, clinical aspects, economics, organizational aspects, and patient aspects.

Conclusions

We have developed an health technology assessment-based framework to support the introduction of AI technologies into healthcare in medical imaging. It is essential to ensure informed and valid decisions regarding the adoption of AI with a structured process and tool. MAS-AI can help support decision making and provide greater transparency for all parties.

Deep Learning Supplants Visual Analysis by Experienced Operators for the Diagnosis of Cardiac Amyloidosis by Cine-CMR

Background: Diagnosing cardiac amyloidosis (CA) from cine-CMR (cardiac magnetic resonance) alone is not reliable. In this study, we tested if a convolutional neural network (CNN) could outperform the visual diagnosis of experienced operators. Method: 119 patients with cardiac amyloidosis and 122 patients with left ventricular hypertrophy (LVH) of other origins were retrospectively selected. Diastolic and systolic cine-CMR images were preprocessed and labeled. A dual-input visual geometry group (VGG ) model was used for binary image classification. All images belonging to the same patient were distributed in the same set. Accuracy and area under the curve (AUC) were calculated per frame and per patient from a 40% held-out test set. Results were compared to a visual analysis assessed by three experienced operators. Results: frame-based comparisons between humans and a CNN provided an accuracy of 0.605 vs. 0.746 (p < 0.0008) and an AUC of 0.630 vs. 0.824 (p < 0.0001). Patient-based comparisons provided an accuracy of 0.660 vs. 0.825 (p < 0.008) and an AUC of 0.727 vs. 0.895 (p < 0.002). Conclusion: based on cine-CMR images alone, a CNN is able to discriminate cardiac amyloidosis from LVH of other origins better than experienced human operators (15 to 20 points more in absolute value for accuracy and AUC), demonstrating a unique capability to identify what the eyes cannot see through classical radiological analysis.

RSNA-MICCAI Panel Discussion: 2. Leveraging the Full Potential of AI-Radiologists and Data Scientists Working Together

In March 2021, the Radiological Society of North America hosted a virtual panel discussion with members of the Medical Image Computing and Computer Assisted Intervention Society. Both organizations share a vision to develop radiologic and medical imaging techniques through advanced quantitative imaging biomarkers and artificial intelligence. The panel addressed how radiologists and data scientists can collaborate to advance the science of AI in radiology. Keywords: Adults and Pediatrics, Segmentation, Feature Detection, Quantification, Diagnosis/Classification, Prognosis/Classification © RSNA, 2021.

Artificial intelligence-based detection of aortic stenosis from chest radiographs

Aims

We aimed to develop models to detect aortic stenosis (AS) from chest radiographs-one of the most basic imaging tests-with artificial intelligence.

Methods and results

We used 10 433 retrospectively collected digital chest radiographs from 5638 patients to train, validate, and test three deep learning models. Chest radiographs were collected from patients who had also undergone echocardiography at a single institution between July 2016 and May 2019. These were labelled from the corresponding echocardiography assessments as AS-positive or AS-negative. The radiographs were separated on a patient basis into training [8327 images from 4512 patients, mean age 65 ±  (standard deviation) 15 years], validation (1041 images from 563 patients, mean age 65 ± 14 years), and test (1065 images from 563 patients, mean age 65 ± 14 years) datasets. The soft voting-based ensemble of the three developed models had the best overall performance for predicting AS with an area under the receiver operating characteristic curve, sensitivity, specificity, accuracy, positive predictive value, and negative predictive value of 0.83 (95% confidence interval 0.77-0.88), 0.78 (0.67-0.86), 0.71 (0.68-0.73), 0.71 (0.68-0.74), 0.18 (0.14-0.23), and 0.97 (0.96-0.98), respectively, in the validation dataset and 0.83 (0.78-0.88), 0.83 (0.74-0.90), 0.69 (0.66-0.72), 0.71 (0.68-0.73), 0.23 (0.19-0.28), and 0.97 (0.96-0.98), respectively, in the test dataset.

Conclusion

Deep learning models using chest radiographs have the potential to differentiate between radiographs of patients with and without AS.

Lay Summary

We created artificial intelligence (AI) models using deep learning to identify aortic stenosis (AS) from chest radiographs. Three AI models were developed and evaluated with 10 433 retrospectively collected radiographs and labelled from echocardiography reports. The ensemble AI model could detect AS in a test dataset with an area under the receiver operating characteristic curve of 0.83 (95% confidence interval 0.78-0.88). Since chest radiography is a cost-effective and widely available imaging test, our model can provide an additive resource for the detection of AS.

Convolutional neural network performance compared to radiologists in detecting intracranial hemorrhage from brain computed tomography: A systematic review and meta-analysis

PURPOSE

To compare the diagnostic accuracy of convolutional neural networks (CNN) with radiologists as the reference standard in the diagnosis of intracranial hemorrhages (ICH) with non contrast computed tomography of the cerebrum (NCTC).

METHODS

PubMed, Embase, Scopus, and Web of Science were searched for the period from 1 January 2012 to 20 July 2020; eligible studies included patients with and without ICH as the target condition undergoing NCTC, studies had deep learning algorithms based on CNNs and radiologists reports as the minimum reference standard. Pooled sensitivities, specificities and a summary receiver operating characteristics curve (SROC) were employed for meta-analysis.

RESULTS

5,119 records were identified through database searching. Title-screening left 47 studies for full-text assessment and 6 studies for meta-analysis. Comparing the CNN performance to reference standards in the retrospective studies found a pooled sensitivity of 96.00% (95% CI: 93.00% to 97.00%), pooled specificity of 97.00% (95% CI: 90.00% to 99.00%) and SROC of 98.00% (95% CI: 97.00% to 99.00%), and combining retrospective and studies with external datasets found a pooled sensitivity of 95.00% (95% CI: 91.00% to 97.00%), pooled specificity of 96.00% (95% CI: 91.00% to 98.00%) and a pooled SROC of 98.00% (95% CI: 97.00% to 99.00%).

CONCLUSION

This review found the diagnostic performance of CNNs to be equivalent to that of radiologists for retrospective studies. Out-of-sample external validation studies pooled with retrospective studies found CNN performance to be slightly worse. There is a critical need for studies with a robust reference standard and external data-set validation.

Diagnostic test accuracy of artificial intelligence analysis of cross-sectional imaging in pulmonary hypertension: a systematic literature review

OBJECTIVES

To undertake the first systematic review examining the performance of artificial intelligence (AI) applied to cross-sectional imaging for the diagnosis of acquired pulmonary arterial hypertension (PAH).

METHODS

Searches of Medline, Embase and Web of Science were undertaken on 1 July 2020. Original publications studying AI applied to cross-sectional imaging for the diagnosis of acquired PAH in adults were identified through two-staged double-blinded review. Study quality was assessed using the Quality Assessment of Diagnostic Accuracy Studies and Checklist for Artificial Intelligence in Medicine frameworks. Narrative synthesis was undertaken following Synthesis Without Meta-Analysis guidelines. This review received no funding and was registered in the International Prospective Register of Systematic Reviews (ID:CRD42020196295).

RESULTS

Searches returned 476 citations. Three retrospective observational studies, published between 2016 and 2020, were selected for data-extraction. Two methods applied to cardiac-MRI demonstrated high diagnostic accuracy, with the best model achieving AUC=0.90 (95% CI: 0.85-0.93), 89% sensitivity and 81% specificity. Stronger results were achieved using cardiac-MRI for classification of idiopathic PAH, achieving AUC=0.97 (95% CI: 0.89-1.0), 96% sensitivity and 87% specificity. One study reporting CT-based AI demonstrated lower accuracy, with 64.6% sensitivity and 97.0% specificity.

CONCLUSIONS

Automated methods for identifying PAH on cardiac-MRI are emerging with high diagnostic accuracy. AI applied to cross-sectional imaging may provide non-invasive support to reduce diagnostic delay in PAH. This would be helped by stronger solutions in other modalities.

ADVANCES IN KNOWLEDGE

There is a significant shortage of research in this important area. Early detection of PAH would be supported by further research advances on the promising emerging technologies identified.

Radiomics Models for Predicting Microvascular Invasion in Hepatocellular Carcinoma: A Systematic Review and Radiomics Quality Score Assessment

Preoperative prediction of microvascular invasion (MVI) is of importance in hepatocellular carcinoma (HCC) patient treatment management. Plenty of radiomics models for MVI prediction have been proposed. This study aimed to elucidate the role of radiomics models in the prediction of MVI and to evaluate their methodological quality. The methodological quality was assessed by the Radiomics Quality Score (RQS), and the risk of bias was evaluated by the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2). Twenty-two studies using CT, MRI, or PET/CT for MVI prediction were included. All were retrospective studies, and only two had an external validation cohort. The AUC values of the prediction models ranged from 0.69 to 0.94 in the test cohort. Substantial methodological heterogeneity existed, and the methodological quality was low, with an average RQS score of 10 (28% of the total). Most studies demonstrated a low or unclear risk of bias in the domains of QUADAS-2. In conclusion, a radiomics model could be an accurate and effective tool for MVI prediction in HCC patients, although the methodological quality has so far been insufficient. Future prospective studies with an external validation cohort in accordance with a standardized radiomics workflow are expected to supply a reliable model that translates into clinical utilization.

Radiomics Models for Predicting Microvascular Invasion in Hepatocellular Carcinoma: A Systematic Review and Radiomics Quality Score Assessment

Preoperative prediction of microvascular invasion (MVI) is of importance in hepatocellular carcinoma (HCC) patient treatment management. Plenty of radiomics models for MVI prediction have been proposed. This study aimed to elucidate the role of radiomics models in the prediction of MVI and to evaluate their methodological quality. The methodological quality was assessed by the Radiomics Quality Score (RQS), and the risk of bias was evaluated by the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2). Twenty-two studies using CT, MRI, or PET/CT for MVI prediction were included. All were retrospective studies, and only two had an external validation cohort. The AUC values of the prediction models ranged from 0.69 to 0.94 in the test cohort. Substantial methodological heterogeneity existed, and the methodological quality was low, with an average RQS score of 10 (28% of the total). Most studies demonstrated a low or unclear risk of bias in the domains of QUADAS-2. In conclusion, a radiomics model could be an accurate and effective tool for MVI prediction in HCC patients, although the methodological quality has so far been insufficient. Future prospective studies with an external validation cohort in accordance with a standardized radiomics workflow are expected to supply a reliable model that translates into clinical utilization.

Deep Learning-Based Computer-Aided Detection System for Automated Treatment Response Assessment of Brain Metastases on 3D MRI

BACKGROUND

Although accurate treatment response assessment for brain metastases (BMs) is crucial, it is highly labor intensive. This retrospective study aimed to develop a computer-aided detection (CAD) system for automated BM detection and treatment response evaluation using deep learning.

METHODS

We included 214 consecutive MRI examinations of 147 patients with BM obtained between January 2015 and August 2016. These were divided into the training (174 MR images from 127 patients) and test datasets according to temporal separation (temporal test set #1; 40 MR images from 20 patients). For external validation, 24 patients with BM and 11 patients without BM from other institutions were included (geographic test set). In addition, we included 12 MRIs from BM patients obtained between August 2017 and March 2020 (temporal test set #2). Detection sensitivity, dice similarity coefficient (DSC) for segmentation, and agreements in one-dimensional and volumetric Response Assessment in Neuro-Oncology Brain Metastases (RANO-BM) criteria between CAD and radiologists were assessed.

RESULTS

In the temporal test set #1, the sensitivity was 75.1% (95% confidence interval [CI]: 69.6%, 79.9%), mean DSC was 0.69 ± 0.22, and false-positive (FP) rate per scan was 0.8 for BM ≥ 5 mm. Agreements in the RANO-BM criteria were moderate (κ, 0.52) and substantial (κ, 0.68) for one-dimensional and volumetric, respectively. In the geographic test set, sensitivity was 87.7% (95% CI: 77.2%, 94.5%), mean DSC was 0.68 ± 0.20, and FP rate per scan was 1.9 for BM ≥ 5 mm. In the temporal test set #2, sensitivity was 94.7% (95% CI: 74.0%, 99.9%), mean DSC was 0.82 ± 0.20, and FP per scan was 0.5 (6/12) for BM ≥ 5 mm.

CONCLUSIONS

Our CAD showed potential for automated treatment response assessment of BM ≥ 5 mm.

Deep-learning-based synthesis of post-contrast T1-weighted MRI for tumour response assessment in neuro-oncology: a multicentre, retrospective cohort study

BACKGROUND

Gadolinium-based contrast agents (GBCAs) are widely used to enhance tissue contrast during MRI scans and play a crucial role in the management of patients with cancer. However, studies have shown gadolinium deposition in the brain after repeated GBCA administration with yet unknown clinical significance. We aimed to assess the feasibility and diagnostic value of synthetic post-contrast T1-weighted MRI generated from pre-contrast MRI sequences through deep convolutional neural networks (dCNN) for tumour response assessment in neuro-oncology.

METHODS

In this multicentre, retrospective cohort study, we used MRI examinations to train and validate a dCNN for synthesising post-contrast T1-weighted sequences from pre-contrast T1-weighted, T2-weighted, and fluid-attenuated inversion recovery sequences. We used MRI scans with availability of these sequences from 775 patients with glioblastoma treated at Heidelberg University Hospital, Heidelberg, Germany (775 MRI examinations); 260 patients who participated in the phase 2 CORE trial (1083 MRI examinations, 59 institutions); and 505 patients who participated in the phase 3 CENTRIC trial (3147 MRI examinations, 149 institutions). Separate training runs to rank the importance of individual sequences and (for a subset) diffusion-weighted imaging were conducted. Independent testing was performed on MRI data from the phase 2 and phase 3 EORTC-26101 trial (521 patients, 1924 MRI examinations, 32 institutions). The similarity between synthetic and true contrast enhancement on post-contrast T1-weighted MRI was quantified using the structural similarity index measure (SSIM). Automated tumour segmentation and volumetric tumour response assessment based on synthetic versus true post-contrast T1-weighted sequences was performed in the EORTC-26101 trial and agreement was assessed with Kaplan-Meier plots.

FINDINGS

The median SSIM score for predicting contrast enhancement on synthetic post-contrast T1-weighted sequences in the EORTC-26101 test set was 0·818 (95% CI 0·817-0·820). Segmentation of the contrast-enhancing tumour from synthetic post-contrast T1-weighted sequences yielded a median tumour volume of 6·31 cm³ (5·60 to 7·14), thereby underestimating the true tumour volume by a median of -0·48 cm³ (-0·37 to -0·76) with the concordance correlation coefficient suggesting a strong linear association between tumour volumes derived from synthetic versus true post-contrast T1-weighted sequences (0·782, 0·751-0·807, p<0·0001). Volumetric tumour response assessment in the EORTC-26101 trial showed a median time to progression of 4·2 months (95% CI 4·1-5·2) with synthetic post-contrast T1-weighted and 4·3 months (4·1-5·5) with true post-contrast T1-weighted sequences (p=0·33). The strength of the association between the time to progression as a surrogate endpoint for predicting the patients' overall survival in the EORTC-26101 cohort was similar when derived from synthetic post-contrast T1-weighted sequences (hazard ratio of 1·749, 95% CI 1·282-2·387, p=0·0004) and model C-index (0·667, 0·622-0·708) versus true post-contrast T1-weighted MRI (1·799, 95% CI 1·314-2·464, p=0·0003) and model C-index (0·673, 95% CI 0·626-0·711).

INTERPRETATION

Generating synthetic post-contrast T1-weighted MRI from pre-contrast MRI using dCNN is feasible and quantification of the contrast-enhancing tumour burden from synthetic post-contrast T1-weighted MRI allows assessment of the patient's response to treatment with no significant difference by comparison with true post-contrast T1-weighted sequences with administration of GBCAs. This finding could guide the application of dCNN in radiology to potentially reduce the necessity of GBCA administration.

FUNDING

Deutsche Forschungsgemeinschaft.

Deep learning-based classification and structure name standardization for organ at risk and target delineations in prostate cancer radiotherapy

Radiotherapy (RT) datasets can suffer from variations in annotation of organ at risk (OAR) and target structures. Annotation standards exist, but their description for prostate targets is limited. This restricts the use of such data for supervised machine learning purposes as it requires properly annotated data. The aim of this work was to develop a modality independent deep learning (DL) model for automatic classification and annotation of prostate RT DICOM structures. Delineated prostate organs at risk (OAR), support- and target structures (gross tumor volume [GTV]/clinical target volume [CTV]/planning target volume [PTV]), along with or without separate vesicles and/or lymph nodes, were extracted as binary masks from 1854 patients. An image modality independent 2D InceptionResNetV2 classification network was trained with varying amounts of training data using four image input channels. Channel 1-3 consisted of orthogonal 2D projections from each individual binary structure. The fourth channel contained a summation of the other available binary structure masks. Structure classification performance was assessed in independent CT (n = 200 pat) and magnetic resonance imaging (MRI) (n = 40 pat) test datasets and an external CT (n = 99 pat) dataset from another clinic. A weighted classification accuracy of 99.4% was achieved during training. The unweighted classification accuracy and the weighted average F1 score among different structures in the CT test dataset were 98.8% and 98.4% and 98.6% and 98.5% for the MRI test dataset, respectively. The external CT dataset yielded the corresponding results 98.4% and 98.7% when analyzed for trained structures only, and results from the full dataset yielded 79.6% and 75.2%. Most misclassifications in the external CT dataset occurred due to multiple CTVs and PTVs being fused together, which was not included in the training data. Our proposed DL-based method for automated renaming and standardization of prostate radiotherapy annotations shows great potential. Clinic specific contouring standards however need to be represented in the training data for successful use. Source code is available at https://github.com/jamtheim/DicomRTStructRenamerPublic.

Automatic segmentation of peripheral arteries and veins in ferumoxytol-enhanced MR angiography

PURPOSE

To automate the segmentation of the peripheral arteries and veins in the lower extremities based on ferumoxytol-enhanced MR angiography (FE-MRA).

METHODS

Our automated pipeline has 2 sequential stages. In the first stage, we used a 3D U-Net with local attention gates, which was trained based on a combination of the Focal Tversky loss with region mutual loss under a deep supervision mechanism to segment the vasculature from the high-resolution FE-MRA datasets. In the second stage, we used time-resolved images to separate the arteries from the veins. Because the ultimate segmentation quality of the arteries and veins relies on the performance of the first stage, we thoroughly evaluated the different aspects of the segmentation network and compared its performance in blood vessel segmentation with currently accepted state-of-the-art networks, including Volumetric-Net, DeepVesselNet-FCN, and Uception.

RESULTS

We achieved a competitive F1 = 0.8087 and recall = 0.8410 for blood vessel segmentation compared with F1 = (0.7604, 0.7573, 0.7651) and recall = (0.7791, 0.7570, 0.7774) obtained with Volumetric-Net, DeepVesselNet-FCN, and Uception. For the artery and vein separation stage, we achieved F1 = (0.8274/0.7863) in the calf region, which is the most challenging region in peripheral arteries and veins segmentation.

CONCLUSION

Our pipeline is capable of fully automatic vessel segmentation based on FE-MRA without need for human interaction in <4 min. This method improves upon manual segmentation by radiologists, which routinely takes several hours.

Radiomics in Oncology: A Practical Guide

Radiomics refers to the extraction of mineable data from medical imaging and has been applied within oncology to improve diagnosis, prognostication, and clinical decision support, with the goal of delivering precision medicine. The authors provide a practical approach for successfully implementing a radiomic workflow from planning and conceptualization through manuscript writing. Applications in oncology typically are either classification tasks that involve computing the probability of a sample belonging to a category, such as benign versus malignant, or prediction of clinical events with a time-to-event analysis, such as overall survival. The radiomic workflow is multidisciplinary, involving radiologists and data and imaging scientists, and follows a stepwise process involving tumor segmentation, image preprocessing, feature extraction, model development, and validation. Images are curated and processed before segmentation, which can be performed on tumors, tumor subregions, or peritumoral zones. Extracted features typically describe the distribution of signal intensities and spatial relationship of pixels within a region of interest. To improve model performance and reduce overfitting, redundant and nonreproducible features are removed. Validation is essential to estimate model performance in new data and can be performed iteratively on samples of the dataset (cross-validation) or on a separate hold-out dataset by using internal or external data. A variety of noncommercial and commercial radiomic software applications can be used. Guidelines and artificial intelligence checklists are useful when planning and writing up radiomic studies. Although interest in the field continues to grow, radiologists should be familiar with potential pitfalls to ensure that meaningful conclusions can be drawn.

Online supplemental material is available for this article.

Published under a CC BY 4.0 license.

Reproducibility standards for machine learning in the life sciences

To make machine learning analyses in the life sciences more computationally reproducible, we propose standards based on data, model, and code publication, programming best practices, and workflow automation. By meeting these standards, the community of researchers applying machine learning methods in the life sciences can ensure that their analyses are worthy of trust.

this article has been peer reviewed.

Systematic review of research design and reporting of imaging studies applying convolutional neural networks for radiological cancer diagnosis

OBJECTIVES

To perform a systematic review of design and reporting of imaging studies applying convolutional neural network models for radiological cancer diagnosis.

METHODS

A comprehensive search of PUBMED, EMBASE, MEDLINE and SCOPUS was performed for published studies applying convolutional neural network models to radiological cancer diagnosis from January 1, 2016, to August 1, 2020. Two independent reviewers measured compliance with the Checklist for Artificial Intelligence in Medical Imaging (CLAIM). Compliance was defined as the proportion of applicable CLAIM items satisfied.

RESULTS

One hundred eighty-six of 655 screened studies were included. Many studies did not meet the criteria for current design and reporting guidelines. Twenty-seven percent of studies documented eligibility criteria for their data (50/186, 95% CI 21-34%), 31% reported demographics for their study population (58/186, 95% CI 25-39%) and 49% of studies assessed model performance on test data partitions (91/186, 95% CI 42-57%). Median CLAIM compliance was 0.40 (IQR 0.33-0.49). Compliance correlated positively with publication year (ρ = 0.15, p = .04) and journal H-index (ρ = 0.27, p < .001). Clinical journals demonstrated higher mean compliance than technical journals (0.44 vs. 0.37, p < .001).

CONCLUSIONS

Our findings highlight opportunities for improved design and reporting of convolutional neural network research for radiological cancer diagnosis.

KEY POINTS

• Imaging studies applying convolutional neural networks (CNNs) for cancer diagnosis frequently omit key clinical information including eligibility criteria and population demographics. • Fewer than half of imaging studies assessed model performance on explicitly unobserved test data partitions. • Design and reporting standards have improved in CNN research for radiological cancer diagnosis, though many opportunities remain for further progress.

Radiomics in PET Imaging:: A Practical Guide for Newcomers

Radiomics has undergone considerable development in recent years. In PET imaging, very promising results concerning the ability of handcrafted features to predict the biological characteristics of lesions and to assess patient prognosis or response to treatment have been reported in the literature. This article presents a checklist for designing a reliable radiomic study, gives an overview of the steps of the pipeline, and outlines approaches for data harmonization. Tips are provided for critical reading of the content of articles. The advantages and limitations of handcrafted radiomics compared with deep-learning approaches for the characterization of PET images are also discussed.

AI in medical physics: guidelines for publication

The Abstract is intended to provide a concise summary of the study and its scientific findings. For AI/ML applications in medical physics, a problem statement and rationale for utilizing these algorithms are necessary while highlighting the novelty of the approach. A brief numerical description of how the data are partitioned into subsets for training of the AI/ML algorithm, validation (including tuning of parameters), and independent testing of algorithm performance is required. This is to be followed by a summary of the results and statistical metrics that quantify the performance of the AI/ML algorithm.

Radiomics, machine learning, and artificial intelligence-what the neuroradiologist needs to know

PURPOSE

Artificial intelligence (AI) is playing an ever-increasing role in Neuroradiology.

METHODS

When designing AI-based research in neuroradiology and appreciating the literature, it is important to understand the fundamental principles of AI. Training, validation, and test datasets must be defined and set apart as priorities. External validation and testing datasets are preferable, when feasible. The specific type of learning process (supervised vs. unsupervised) and the machine learning model also require definition. Deep learning (DL) is an AI-based approach that is modelled on the structure of neurons of the brain; convolutional neural networks (CNN) are a commonly used example in neuroradiology.

RESULTS

Radiomics is a frequently used approach in which a multitude of imaging features are extracted from a region of interest and subsequently reduced and selected to convey diagnostic or prognostic information. Deep radiomics uses CNNs to directly extract features and obviate the need for predefined features.

CONCLUSION

Common limitations and pitfalls in AI-based research in neuroradiology are limited sample sizes ("small-n-large-p problem"), selection bias, as well as overfitting and underfitting.

Toward High-Throughput Artificial Intelligence-Based Segmentation in Oncological PET Imaging

Artificial intelligence (AI) techniques for image-based segmentation have garnered much attention in recent years. Convolutional neural networks have shown impressive results and potential toward fully automated segmentation in medical imaging, and particularly PET imaging. To cope with the limited access to annotated data needed in supervised AI methods, given tedious and prone-to-error manual delineations, semi-supervised and unsupervised AI techniques have also been explored for segmentation of tumors or normal organs in single- and bimodality scans. This work reviews existing AI techniques for segmentation tasks and the evaluation criteria for translational AI-based segmentation efforts toward routine adoption in clinical workflows.

Radiomics and Machine Learning Can Differentiate Transient Osteoporosis from Avascular Necrosis of the Hip

Differentiation between transient osteoporosis (TOH) and avascular necrosis (AVN) of the hip is a longstanding challenge in musculoskeletal radiology. The purpose of this study was to utilize MRI-based radiomics and machine learning (ML) for accurate differentiation between the two entities. A total of 109 hips with TOH and 104 hips with AVN were retrospectively included. Femoral heads and necks with segmented radiomics features were extracted. Three ML classifiers (XGboost, CatBoost and SVM) using 38 relevant radiomics features were trained on 70% and validated on 30% of the dataset. ML performance was compared to two musculoskeletal radiologists, a general radiologist and two radiology residents. XGboost achieved the best performance with an area under the curve (AUC) of 93.7% (95% CI from 87.7 to 99.8%) among ML models. MSK radiologists achieved an AUC of 90.6% (95% CI from 86.7% to 94.5%) and 88.3% (95% CI from 84% to 92.7%), respectively, similar to residents. The general radiologist achieved an AUC of 84.5% (95% CI from 80% to 89%), significantly lower than of XGboost (p = 0.017). In conclusion, radiomics-based ML achieved a performance similar to MSK radiologists and significantly higher compared to general radiologists in differentiating between TOH and AVN.

Accelerating antibiotic discovery through artificial intelligence

By targeting invasive organisms, antibiotics insert themselves into the ancient struggle of the host-pathogen evolutionary arms race. As pathogens evolve tactics for evading antibiotics, therapies decline in efficacy and must be replaced, distinguishing antibiotics from most other forms of drug development. Together with a slow and expensive antibiotic development pipeline, the proliferation of drug-resistant pathogens drives urgent interest in computational methods that promise to expedite candidate discovery. Strides in artificial intelligence (AI) have encouraged its application to multiple dimensions of computer-aided drug design, with increasing application to antibiotic discovery. This review describes AI-facilitated advances in the discovery of both small molecule antibiotics and antimicrobial peptides. Beyond the essential prediction of antimicrobial activity, emphasis is also given to antimicrobial compound representation, determination of drug-likeness traits, antimicrobial resistance, and de novo molecular design. Given the urgency of the antimicrobial resistance crisis, we analyze uptake of open science best practices in AI-driven antibiotic discovery and argue for openness and reproducibility as a means of accelerating preclinical research. Finally, trends in the literature and areas for future inquiry are discussed, as artificially intelligent enhancements to drug discovery at large offer many opportunities for future applications in antibiotic development.

Artificial intelligence in radiation oncology: A review of its current status and potential application for the radiotherapy workforce

OBJECTIVE

Radiation oncology is a continually evolving speciality. With the development of new imaging modalities and advanced imaging processing techniques, there is an increasing amount of data available to practitioners. In this narrative review, Artificial Intelligence (AI) is used as a reference to machine learning, and its potential, along with current problems in the field of radiation oncology, are considered from a technical position.

KEY FINDINGS

AI has the potential to harness the availability of data for improving patient outcomes, reducing toxicity, and easing clinical burdens. However, problems including the requirement of complexity of data, undefined core outcomes and limited generalisability are apparent.

CONCLUSION

This original review highlights considerations for the radiotherapy workforce, particularly therapeutic radiographers, as there will be an increasing requirement for their familiarity with AI due to their unique position as the interface between imaging technology and patients.

IMPLICATIONS FOR PRACTICE

Collaboration between AI experts and the radiotherapy workforce are required to overcome current issues before clinical adoption. The development of educational resources and standardised reporting of AI studies may help facilitate this.

Deep Learning-Based Natural Language Processing in Radiology: The Impact of Report Complexity, Disease Prevalence, Dataset Size, and Algorithm Type on Model Performance

In radiology, natural language processing (NLP) allows the extraction of valuable information from radiology reports. It can be used for various downstream tasks such as quality improvement, epidemiological research, and monitoring guideline adherence. Class imbalance, variation in dataset size, variation in report complexity, and algorithm type all influence NLP performance but have not yet been systematically and interrelatedly evaluated. In this study, we investigate these factors on the performance of four types [a fully connected neural network (Dense), a long short-term memory recurrent neural network (LSTM), a convolutional neural network (CNN), and a Bidirectional Encoder Representations from Transformers (BERT)] of deep learning-based NLP. Two datasets consisting of radiologist-annotated reports of both trauma radiographs (n = 2469) and chest radiographs and computer tomography (CT) studies (n = 2255) were split into training sets (80%) and testing sets (20%). The training data was used as a source to train all four model types in 84 experiments (Fracture-data) and 45 experiments (Chest-data) with variation in size and prevalence. The performance was evaluated on sensitivity, specificity, positive predictive value, negative predictive value, area under the curve, and F score. After the NLP of radiology reports, all four model-architectures demonstrated high performance with metrics up to > 0.90. CNN, LSTM, and Dense were outperformed by the BERT algorithm because of its stable results despite variation in training size and prevalence. Awareness of variation in prevalence is warranted because it impacts sensitivity and specificity in opposite directions.

AI-enhanced breast imaging: Where are we and where are we heading?

Significant advances in imaging analysis and the development of high-throughput methods that can extract and correlate multiple imaging parameters with different clinical outcomes have led to a new direction in medical research. Radiomics and artificial intelligence (AI) studies are rapidly evolving and have many potential applications in breast imaging, such as breast cancer risk prediction, lesion detection and classification, radiogenomics, and prediction of treatment response and clinical outcomes. AI has been applied to different breast imaging modalities, including mammography, ultrasound, and magnetic resonance imaging, in different clinical scenarios. The application of AI tools in breast imaging has an unprecedented opportunity to better derive clinical value from imaging data and reshape the way we care for our patients. The aim of this study is to review the current knowledge and future applications of AI-enhanced breast imaging in clinical practice.

Clinician-driven artificial intelligence in ophthalmology: resources enabling democratization

PURPOSE OF REVIEW

This article aims to discuss the current state of resources enabling the democratization of artificial intelligence (AI) in ophthalmology.

RECENT FINDINGS

Open datasets, efficient labeling techniques, code-free automated machine learning (AutoML) and cloud-based platforms for deployment are resources that enable clinicians with scarce resources to drive their own AI projects.

SUMMARY

Clinicians are the use-case experts who are best suited to drive AI projects tackling patient-relevant outcome measures. Taken together, open datasets, efficient labeling techniques, code-free AutoML and cloud platforms break the barriers for clinician-driven AI. As AI becomes increasingly democratized through such tools, clinicians and patients stand to benefit greatly.

Radiomics and machine learning applications in rectal cancer: Current update and future perspectives

The high incidence of rectal cancer in both sexes makes it one of the most common tumors, with significant morbidity and mortality rates. To define the best treatment option and optimize patient outcome, several rectal cancer biological variables must be evaluated. Currently, medical imaging plays a crucial role in the characterization of this disease, and it often requires a multimodal approach. Magnetic resonance imaging is the first-choice imaging modality for local staging and restaging and can be used to detect high-risk prognostic factors. Computed tomography is widely adopted for the detection of distant metastases. However, conventional imaging has recognized limitations, and many rectal cancer characteristics remain assessable only after surgery and histopathology evaluation. There is a growing interest in artificial intelligence applications in medicine, and imaging is by no means an exception. The introduction of radiomics, which allows the extraction of quantitative features that reflect tumor heterogeneity, allows the mining of data in medical images and paved the way for the identification of potential new imaging biomarkers. To manage such a huge amount of data, the use of machine learning algorithms has been proposed. Indeed, without prior explicit programming, they can be employed to build prediction models to support clinical decision making. In this review, current applications and future perspectives of artificial intelligence in medical imaging of rectal cancer are presented, with an imaging modality-based approach and a keen eye on unsolved issues. The results are promising, but the road ahead for translation in clinical practice is rather long.

Review of study reporting guidelines for clinical studies using artificial intelligence in healthcare

High-quality research is essential in guiding evidence-based care, and should be reported in a way that is reproducible, transparent and where appropriate, provide sufficient detail for inclusion in future meta-analyses. Reporting guidelines for various study designs have been widely used for clinical (and preclinical) studies, consisting of checklists with a minimum set of points for inclusion. With the recent rise in volume of research using artificial intelligence (AI), additional factors need to be evaluated, which do not neatly conform to traditional reporting guidelines (eg, details relating to technical algorithm development). In this review, reporting guidelines are highlighted to promote awareness of essential content required for studies evaluating AI interventions in healthcare. These include published and in progress extensions to well-known reporting guidelines such as Standard Protocol Items: Recommendations for Interventional Trials-AI (study protocols), Consolidated Standards of Reporting Trials-AI (randomised controlled trials), Standards for Reporting of Diagnostic Accuracy Studies-AI (diagnostic accuracy studies) and Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis-AI (prediction model studies). Additionally there are a number of guidelines that consider AI for health interventions more generally (eg, Checklist for Artificial Intelligence in Medical Imaging (CLAIM), minimum information (MI)-CLAIM, MI for Medical AI Reporting) or address a specific element such as the ‘learning curve’ (Developmental and Exploratory Clinical Investigation of Decision-AI). Economic evaluation of AI health interventions is not currently addressed, and may benefit from extension to an existing guideline. In the face of a rapid influx of studies of AI health interventions, reporting guidelines help ensure that investigators and those appraising studies consider both the well-recognised elements of good study design and reporting, while also adequately addressing new challenges posed by AI-specific elements.

Effects of Interobserver Variability on 2D and 3D CT- and MRI-Based Texture Feature Reproducibility of Cartilaginous Bone Tumors

This study aims to investigate the influence of interobserver manual segmentation variability on the reproducibility of 2D and 3D unenhanced computed tomography (CT)- and magnetic resonance imaging (MRI)-based texture analysis. Thirty patients with cartilaginous bone tumors (10 enchondromas, 10 atypical cartilaginous tumors, 10 chondrosarcomas) were retrospectively included. Three radiologists independently performed manual contour-focused segmentation on unenhanced CT and T1-weighted and T2-weighted MRI by drawing both a 2D region of interest (ROI) on the slice showing the largest tumor area and a 3D ROI including the whole tumor volume. Additionally, a marginal erosion was applied to both 2D and 3D segmentations to evaluate the influence of segmentation margins. A total of 783 and 1132 features were extracted from original and filtered 2D and 3D images, respectively. Intraclass correlation coefficient ≥ 0.75 defined feature stability. In 2D vs. 3D contour-focused segmentation, the rates of stable features were 74.71% vs. 86.57% (p < 0.001), 77.14% vs. 80.04% (p = 0.142), and 95.66% vs. 94.97% (p = 0.554) for CT and T1-weighted and T2-weighted images, respectively. Margin shrinkage did not improve 2D (p = 0.343) and performed worse than 3D (p < 0.001) contour-focused segmentation in terms of feature stability. In 2D vs. 3D contour-focused segmentation, matching stable features derived from CT and MRI were 65.8% vs. 68.7% (p = 0.191), and those derived from T1-weighted and T2-weighted images were 76.0% vs. 78.2% (p = 0.285). 2D and 3D radiomic features of cartilaginous bone tumors extracted from unenhanced CT and MRI are reproducible, although some degree of interobserver segmentation variability highlights the need for reliability analysis in future studies.

Deep learning convolutional neural network (DLCNN): unleashing the potential of 18F-FDG PET/CT in lymphoma

This perspective briefly reviewed the applications of 18F-FDG PET/CT in the clinical management of lymphoma and the need for lesion segmentation in those applications. It discussed the limitations of existing segmentation technologies and the great potential of using deep learning convolutional neural network (DLCNN) to accomplish automatic lymphoma segmentation and characterizations. Finally, the authors shared perspectives on the technical challenges that need to be addressed to fully unleash the potential of DLCNN and 18F-FDG PET/CT in the diagnosis, prognosis, and treatment of lymphoma.

Deep learning in knee imaging: a systematic review utilizing a Checklist for Artificial Intelligence in Medical Imaging (CLAIM)

PURPOSE

Our purposes were (1) to explore the methodologic quality of the studies on the deep learning in knee imaging with CLAIM criterion and (2) to offer our vision for the development of CLAIM to assure high-quality reports about the application of AI to medical imaging in knee joint.

MATERIALS AND METHODS

A Checklist for Artificial Intelligence in Medical Imaging systematic review was conducted from January 1, 2015, to June 1, 2020, using PubMed, EMBASE, and Web of Science databases. A total of 36 articles discussing deep learning applications in knee joint imaging were identified, divided by imaging modality, and characterized by imaging task, data source, algorithm type, and outcome metrics.

RESULTS

A total of 36 studies were identified and divided into: X-ray (44.44%) and MRI (55.56%). The mean CLAIM score of the 36 studies was 27.94 (standard deviation, 4.26), which was 66.53% of the ideal score of 42.00. The CLAIM items achieved an average good inter-rater agreement (ICC 0.815, 95% CI 0.660-0.902). In total, 32 studies performed internal cross-validation on the data set, while only 4 studies conducted external validation of the data set.

CONCLUSIONS

The overall scientific quality of deep learning in knee imaging is insufficient; however, deep learning remains a promising technology for diagnostic or predictive purpose. Improvements in study design, validation, and open science need to be made to demonstrate the generalizability of findings and to achieve clinical applications. Widespread application, pre-trained scoring procedure, and modification of CLAIM in response to clinical needs are necessary in the future.

KEY POINTS

• Limited deep learning studies were established in knee imaging with mean score of 27.94, which was 66.53% of the ideal score of 42.00, commonly due to invalidated results, retrospective study design, and absence of a clear definition of the CLAIM items in detail. • A previous trained data extraction instrument allowed reaching moderate inter-rater agreement in the application of the CLAIM, while CLAIM still needs improvement in scoring items and result reporting to become a wide adaptive tool in reviews of deep learning studies.

Artificial Intelligence in Imaging of Chronic Liver Diseases: Current Update and Future Perspectives

Here we review artificial intelligence (AI) models which aim to assess various aspects of chronic liver disease. Despite the clinical importance of hepatocellular carcinoma in the setting of chronic liver disease, we focus this review on AI models which are not lesion-specific and instead review models developed for liver parenchyma segmentation, evaluation of portal circulation, assessment of hepatic fibrosis, and identification of hepatic steatosis. Optimization of these models offers the opportunity to potentially reduce the need for invasive procedures such as catheterization to measure hepatic venous pressure gradient or biopsy to assess fibrosis and steatosis. We compare the performance of these AI models amongst themselves as well as to radiomics approaches and alternate modality assessments. We conclude that these models show promising performance and merit larger-scale evaluation. We review artificial intelligence models that aim to assess various aspects of chronic liver disease aside from hepatocellular carcinoma. We focus this review on models for liver parenchyma segmentation, evaluation of portal circulation, assessment of hepatic fibrosis, and identification of hepatic steatosis. We conclude that these models show promising performance and merit a larger scale evaluation.

Prospective Deployment of Deep Learning in MRI: A Framework for Important Considerations, Challenges, and Recommendations for Best Practices

Artificial intelligence algorithms based on principles of deep learning (DL) have made a large impact on the acquisition, reconstruction, and interpretation of MRI data. Despite the large number of retrospective studies using DL, there are fewer applications of DL in the clinic on a routine basis. To address this large translational gap, we review the recent publications to determine three major use cases that DL can have in MRI, namely, that of model-free image synthesis, model-based image reconstruction, and image or pixel-level classification. For each of these three areas, we provide a framework for important considerations that consist of appropriate model training paradigms, evaluation of model robustness, downstream clinical utility, opportunities for future advances, as well recommendations for best current practices. We draw inspiration for this framework from advances in computer vision in natural imaging as well as additional healthcare fields. We further emphasize the need for reproducibility of research studies through the sharing of datasets and software. LEVEL OF EVIDENCE: 5 TECHNICAL EFFICACY STAGE: 2.

Artificial intelligence in CT for quantifying lung changes in the era of CFTR modulators

RATIONALE

Chest computed tomography (CT) remains the imaging standard for demonstrating cystic fibrosis airway structural disease in vivo. However, visual scorings as an outcome measure are time-consuming, require training, and lack high reproducibility.

OBJECTIVE

To validate a fully automated artificial intelligence-driven scoring of cystic fibrosis lung disease severity.

METHODS

Data were retrospectively collected in three cystic fibrosis reference centers, between 2008 and 2020, in 184 patients 4 to 54-years-old. An algorithm using three two-dimensional convolutional neural networks was trained with 78 patients' CTs (23 530 CT slices) for the semantic labeling of bronchiectasis, peribronchial thickening, bronchial mucus, bronchiolar mucus, and collapse/consolidation. 36 patients' CTs (11 435 CT slices) were used for testing versus ground-truth labels. The method's clinical validity was assessed in an independent group of 70 patients with or without lumacaftor/ivacaftor treatment (n=10 and 60, respectively) with repeat examinations. Similarity and reproducibility were assessed using Dice coefficient, correlations using Spearman test, and paired comparisons using Wilcoxon rank test.

MEASUREMENT AND MAIN RESULTS

The overall pixelwise similarity of artificial intelligence-driven versus ground-truth labels was good (Dice coefficient=0.71). All artificial intelligence-driven volumetric quantifications had moderate to very good correlations to a visual imaging scoring (p<0.001) and fair to good correlations to FEV1% at pulmonary function test (p<0.001). Significant decreases in peribronchial thickening (p=0.005), bronchial mucus (p=0.005), bronchiolar mucus (p=0.007) volumes were measured in patients with lumacaftor/ivacaftor. Conversely, bronchiectasis (p=0.002) and peribronchial thickening (p=0.008) volumes increased in patients without lumacaftor/ivacaftor. The reproducibility was almost perfect (Dice>0.99).

CONCLUSION

Artificial intelligence allows a fully automated volumetric quantification of cystic fibrosis-related modifications over an entire lung. The novel scoring system could provide a robust disease outcome in the era of effective CFTR modulator therapy.

Adoption of artificial intelligence in breast imaging: evaluation, ethical constraints and limitations

Retrospective studies have shown artificial intelligence (AI) algorithms can match as well as enhance radiologist's performance in breast screening. These tools can facilitate tasks not feasible by humans such as the automatic triage of patients and prediction of treatment outcomes. Breast imaging faces growing pressure with the exponential growth in imaging requests and a predicted reduced workforce to provide reports. Solutions to alleviate these pressures are being sought with an increasing interest in the adoption of AI to improve workflow efficiency as well as patient outcomes. Vast quantities of data are needed to test and monitor AI algorithms before and after their incorporation into healthcare systems. Availability of data is currently limited, although strategies are being devised to harness the data that already exists within healthcare institutions. Challenges that underpin the realisation of AI into everyday breast imaging cannot be underestimated and the provision of guidance from national agencies to tackle these challenges, taking into account views from a societal, industrial and healthcare prospective is essential. This review provides background on the evaluation and use of AI in breast imaging in addition to exploring key ethical, technical, legal and regulatory challenges that have been identified so far.

Deep learning-assisted ultra-fast/low-dose whole-body PET/CT imaging

PURPOSE

Tendency is to moderate the injected activity and/or reduce acquisition time in PET examinations to minimize potential radiation hazards and increase patient comfort. This work aims to assess the performance of regular full-dose (FD) synthesis from fast/low-dose (LD) whole-body (WB) PET images using deep learning techniques.

METHODS

Instead of using synthetic LD scans, two separate clinical WB 18F-Fluorodeoxyglucose (18F-FDG) PET/CT studies of 100 patients were acquired: one regular FD (~ 27 min) and one fast or LD (~ 3 min) consisting of 1/8th of the standard acquisition time. A modified cycle-consistent generative adversarial network (CycleGAN) and residual neural network (ResNET) models, denoted as CGAN and RNET, respectively, were implemented to predict FD PET images. The quality of the predicted PET images was assessed by two nuclear medicine physicians. Moreover, the diagnostic quality of the predicted PET images was evaluated using a pass/fail scheme for lesion detectability task. Quantitative analysis using established metrics including standardized uptake value (SUV) bias was performed for the liver, left/right lung, brain, and 400 malignant lesions from the test and evaluation datasets.

RESULTS

CGAN scored 4.92 and 3.88 (out of 5) (adequate to good) for brain and neck + trunk, respectively. The average SUV bias calculated over normal tissues was 3.39 ± 0.71% and - 3.83 ± 1.25% for CGAN and RNET, respectively. Bland-Altman analysis reported the lowest SUV bias (0.01%) and 95% confidence interval of - 0.36, + 0.47 for CGAN compared with the reference FD images for malignant lesions.

CONCLUSION

CycleGAN is able to synthesize clinical FD WB PET images from LD images with 1/8th of standard injected activity or acquisition time. The predicted FD images present almost similar performance in terms of lesion detectability, qualitative scores, and quantification bias and variance.

Chest x-ray automated triage: A semiologic approach designed for clinical implementation, exploiting different types of labels through a combination of four Deep Learning architectures

BACKGROUND AND OBJECTIVES

The multiple chest x-ray datasets released in the last years have ground-truth labels intended for different computer vision tasks, suggesting that performance in automated chest x-ray interpretation might improve by using a method that can exploit diverse types of annotations. This work presents a Deep Learning method based on the late fusion of different convolutional architectures, that allows training with heterogeneous data with a simple implementation, and evaluates its performance on independent test data. We focused on obtaining a clinically useful tool that could be successfully integrated into a hospital workflow.

MATERIALS AND METHODS

Based on expert opinion, we selected four target chest x-ray findings, namely lung opacities, fractures, pneumothorax and pleural effusion. For each finding we defined the most suitable type of ground-truth label, and built four training datasets combining images from public chest x-ray datasets and our institutional archive. We trained four different Deep Learning architectures and combined their outputs with a late fusion strategy, obtaining a unified tool. The performance was measured on two test datasets: an external openly-available dataset, and a retrospective institutional dataset, to estimate performance on the local population.

RESULTS

The external and local test sets had 4376 and 1064 images, respectively, for which the model showed an area under the Receiver Operating Characteristics curve of 0.75 (95%CI: 0.74-0.76) and 0.87 (95%CI: 0.86-0.89) in the detection of abnormal chest x-rays. For the local population, a sensitivity of 86% (95%CI: 84-90), and a specificity of 88% (95%CI: 86-90) were obtained, with no significant differences between demographic subgroups. We present examples of heatmaps to show the accomplished level of interpretability, examining true and false positives.

CONCLUSION

This study presents a new approach for exploiting heterogeneous labels from different chest x-ray datasets, by choosing Deep Learning architectures according to the radiological characteristics of each pathological finding. We estimated the tool's performance on the local population, obtaining results comparable to state-of-the-art metrics. We believe this approach is closer to the actual reading process of chest x-rays by professionals, and therefore more likely to be successful in a real clinical setting.

Machine learning identifies factors related to early joint space narrowing in dysplastic and non-dysplastic hips

OBJECTIVES

To utilise machine learning, unsupervised clustering and multivariate modelling in order to predict severe early joint space narrowing (JSN) from anatomical hip parameters while identifying factors related to joint space width (JSW) in dysplastic and non-dysplastic hips.

METHODS

A total of 507 hip CT examinations of patients 20-55 years old were retrospectively examined, and JSW, center-edge (CE) angle, alpha angle, anterior acetabular sector angle (AASA), and neck-shaft angle (NSA) were recorded. Dysplasia and severe JSN were defined with CE angle < 25^o and JSW< 2 mm, respectively. A random forest classifier was developed to predict severe JSN based on anatomical and demographical data. Multivariate linear regression and two-step unsupervised clustering were performed to identify factors linked to JSW.

RESULTS

In dysplastic hips, lateral or anterior undercoverage alone was not correlated to JSN. AASA (p < 0.005) and CE angle (p < 0.032) were the only factors significantly correlated with JSN in dysplastic hips. In non-dysplastic hips, JSW was inversely correlated to CE angle, AASA, and age and positively correlated to NSA (p < 0.001). A random forest classifier predicted severe JSN (AUC 69.9%, 95%CI 47.9-91.8%). TwoStep cluster modelling identified two distinct patient clusters one with low and one with normal JSW and different anatomical characteristics.

CONCLUSION

Machine learning predicted severe JSN and identified population characteristics related to normal and abnormal joint space width. Dysplasia in one plane was found to be insufficient to cause JSN, highlighting the need for hip anatomy assessment on multiple planes.

KEY POINTS

• Neither anterior nor lateral acetabular dysplasia was sufficient to independently reduce joint space width in a multivariate linear regression model of dysplastic hips. • A random forest classifier was developed based on measurements and demographic parameters from 507 hip joints, achieving an area under the curve of 69.9% in the external validation set, in predicting severe joint space narrowing based on anatomical hip parameters and age. • Unsupervised TwoStep cluster analysis revealed two distinct population groups, one with low and one with normal joint space width, characterised by differences in hip morphology.

CT and MRI radiomics of bone and soft-tissue sarcomas: a systematic review of reproducibility and validation strategies

Background

Feature reproducibility and model validation are two main challenges of radiomics. This study aims to systematically review radiomic feature reproducibility and predictive model validation strategies in studies dealing with CT and MRI radiomics of bone and soft-tissue sarcomas. The ultimate goal is to promote achieving a consensus on these aspects in radiomic workflows and facilitate clinical transferability.

Results

Out of 278 identified papers, forty-nine papers published between 2008 and 2020 were included. They dealt with radiomics of bone (n = 12) or soft-tissue (n = 37) tumors. Eighteen (37%) studies included a feature reproducibility analysis. Inter-/intra-reader segmentation variability was the theme of reproducibility analysis in 16 (33%) investigations, outnumbering the analyses focused on image acquisition or post-processing (n = 2, 4%). The intraclass correlation coefficient was the most commonly used statistical method to assess reproducibility, which ranged from 0.6 and 0.9. At least one machine learning validation technique was used for model development in 25 (51%) papers, and K-fold cross-validation was the most commonly employed. A clinical validation of the model was reported in 19 (39%) papers. It was performed using a separate dataset from the primary institution (i.e., internal validation) in 14 (29%) studies and an independent dataset related to different scanners or from another institution (i.e., independent validation) in 5 (10%) studies.

Conclusions

The issues of radiomic feature reproducibility and model validation varied largely among the studies dealing with musculoskeletal sarcomas and should be addressed in future investigations to bring the field of radiomics from a preclinical research area to the clinical stage.