Checklist for Artificial Intelligence in Medical Imaging (CLAIM): A Guide for Authors and Reviewers

Quality of Multicenter Studies Using MRI Radiomics for Diagnosing Clinically Significant Prostate Cancer: A Systematic Review

Background: Reproducibility and generalization are major challenges for clinically significant prostate cancer modeling using MRI radiomics. Multicenter data seem indispensable to deal with these challenges, but the quality of such studies is currently unknown. The aim of this study was to systematically review the quality of multicenter studies on MRI radiomics for diagnosing clinically significant PCa. Methods: This systematic review followed the 2020 Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist. Multicenter studies investigating the value of MRI radiomics for the diagnosis of clinically significant prostate cancer were included. Quality was assessed using the checklist for artificial intelligence in medical imaging (CLAIM) and the radiomics quality score (RQS). CLAIM consisted of 42 equally important items referencing different elements of good practice AI in medical imaging. RQS consisted of 36 points awarded over 16 items related to good practice radiomics. Final CLAIM and RQS scores were percentage-based, allowing for a total quality score consisting of the average of CLAIM and RQS. Results: Four studies were included. The average total CLAIM score was 74.6% and the average RQS was 52.8%. The corresponding average total quality score (CLAIM + RQS) was 63.7%. Conclusions: A very small number of multicenter radiomics PCa classification studies have been performed with the existing studies being of bad or average quality. Good multicenter studies might increase by encouraging preferably prospective data sharing and paying extra care to documentation in regards to reproducibility and clinical utility.

Hyperparameter Tuning and Automatic Image Augmentation for Deep Learning-Based Angle Classification on Intraoral Photographs-A Retrospective Study

We aimed to assess the effects of hyperparameter tuning and automatic image augmentation for deep learning-based classification of orthodontic photographs along the Angle classes. Our dataset consisted of 605 images of Angle class I, 1038 images of class II, and 408 images of class III. We trained ResNet architectures for classification of different combinations of learning rate and batch size. For the best combination, we compared the performance of models trained with and without automatic augmentation using 10-fold cross-validation. We used GradCAM to increase explainability, which can provide heat maps containing the salient areas relevant for the classification. The best combination of hyperparameters yielded a model with an accuracy of 0.63-0.64, F1-score 0.61-0.62, sensitivity 0.59-0.65, and specificity 0.80-0.81. For all metrics, it was apparent that there was an ideal corridor of batch size and learning rate combinations; smaller learning rates were associated with higher classification performance. Overall, the performance was highest for learning rates of around 1-3 × 10-6 and a batch size of eight, respectively. Additional automatic augmentation improved all metrics by 5-10% for all metrics. Misclassifications were most common between Angle classes I and II. GradCAM showed that the models employed features relevant for human classification, too. The choice of hyperparameters drastically affected the performance of deep learning models in orthodontics, and automatic image augmentation resulted in further improvements. Our models managed to classify the dental sagittal occlusion along Angle classes based on digital intraoral photos.

Current challenges of implementing artificial intelligence in medical imaging

The idea of using artificial intelligence (AI) in medical practice has gained vast interest due to its potential to revolutionise healthcare systems. However, only some AI algorithms are utilised due to systems' uncertainties, besides the never-ending list of ethical and legal concerns. This paper intends to provide an overview of current AI challenges in medical imaging with an ultimate aim to foster better and effective communication among various stakeholders to encourage AI technology development. We identify four main challenges in implementing AI in medical imaging, supported with consequences and past events when these problems fail to mitigate. Among them is the creation of a robust AI algorithm that is fair, trustable and transparent. Another issue is on data governance, in which best practices in data sharing must be established to promote trust and protect the patients' privacy. Next, stakeholders, such as the government, technology companies and hospital management, should come to a consensus in creating trustworthy AI policies and regulatory frameworks, which is the fourth challenge, to support, encourage and spur innovation in digital AI healthcare technology. Lastly, we discussed the efforts of various organizations such as the World Health Organisation (WHO), American College of Radiology (ACR), European Society of Radiology (ESR) and Radiological Society of North America (RSNA), who are already actively pursuing ethical developments in AI. The efforts by various stakeholders will eventually overcome hurdles and the deployment of AI-driven healthcare applications in clinical practice will become a reality and hence lead to better healthcare services and outcomes.

Research in Musculoskeletal Radiology: Setting Goals and Strategic Directions

The future of musculoskeletal (MSK) radiology is being built on research developments in the field. Over the past decade, MSK imaging research has been dominated by advancements in molecular imaging biomarkers, artificial intelligence, radiomics, and novel high-resolution equipment. Adequate preparation of trainees and specialists will ensure that current and future leaders will be prepared to embrace and critically appraise technological developments, will be up to date on clinical developments, such as the use of artificial tissues, will define research directions, and will actively participate and lead multidisciplinary research. This review presents an overview of the current MSK research landscape and proposes tangible future goals and strategic directions that will fortify the future of MSK radiology.

Artificial intelligence for radiological paediatric fracture assessment: a systematic review

BACKGROUND

Majority of research and commercial efforts have focussed on use of artificial intelligence (AI) for fracture detection in adults, despite the greater long-term clinical and medicolegal implications of missed fractures in children. The objective of this study was to assess the available literature regarding diagnostic performance of AI tools for paediatric fracture assessment on imaging, and where available, how this compares with the performance of human readers.

MATERIALS AND METHODS

MEDLINE, Embase and Cochrane Library databases were queried for studies published between 1 January 2011 and 2021 using terms related to 'fracture', 'artificial intelligence', 'imaging' and 'children'. Risk of bias was assessed using a modified QUADAS-2 tool. Descriptive statistics for diagnostic accuracies were collated.

RESULTS

Nine eligible articles from 362 publications were included, with most (8/9) evaluating fracture detection on radiographs, with the elbow being the most common body part. Nearly all articles used data derived from a single institution, and used deep learning methodology with only a few (2/9) performing external validation. Accuracy rates generated by AI ranged from 88.8 to 97.9%. In two of the three articles where AI performance was compared to human readers, sensitivity rates for AI were marginally higher, but this was not statistically significant.

CONCLUSIONS

Wide heterogeneity in the literature with limited information on algorithm performance on external datasets makes it difficult to understand how such tools may generalise to a wider paediatric population. Further research using a multicentric dataset with real-world evaluation would help to better understand the impact of these tools.

Semantic segmentation of multispectral photoacoustic images using deep learning

Photoacoustic (PA) imaging has the potential to revolutionize functional medical imaging in healthcare due to the valuable information on tissue physiology contained in multispectral photoacoustic measurements. Clinical translation of the technology requires conversion of the high-dimensional acquired data into clinically relevant and interpretable information. In this work, we present a deep learning-based approach to semantic segmentation of multispectral photoacoustic images to facilitate image interpretability. Manually annotated photoacoustic and ultrasound imaging data are used as reference and enable the training of a deep learning-based segmentation algorithm in a supervised manner. Based on a validation study with experimentally acquired data from 16 healthy human volunteers, we show that automatic tissue segmentation can be used to create powerful analyses and visualizations of multispectral photoacoustic images. Due to the intuitive representation of high-dimensional information, such a preprocessing algorithm could be a valuable means to facilitate the clinical translation of photoacoustic imaging.

Conventional and artificial intelligence-based imaging for biomarker discovery in chronic liver disease

Chronic liver diseases, resulting from chronic injuries of various causes, lead to cirrhosis with life-threatening complications including liver failure, portal hypertension, hepatocellular carcinoma. A key unmet medical need is robust non-invasive biomarkers to predict patient outcome, stratify patients for risk of disease progression and monitor response to emerging therapies. Quantitative imaging biomarkers have already been developed, for instance, liver elastography for staging fibrosis or proton density fat fraction on magnetic resonance imaging for liver steatosis. Yet, major improvements, in the field of image acquisition and analysis, are still required to be able to accurately characterize the liver parenchyma, monitor its changes and predict any pejorative evolution across disease progression. Artificial intelligence has the potential to augment the exploitation of massive multi-parametric data to extract valuable information and achieve precision medicine. Machine learning algorithms have been developed to assess non-invasively certain histological characteristics of chronic liver diseases, including fibrosis and steatosis. Although still at an early stage of development, artificial intelligence-based imaging biomarkers provide novel opportunities to predict the risk of progression from early-stage chronic liver diseases toward cirrhosis-related complications, with the ultimate perspective of precision medicine. This review provides an overview of emerging quantitative imaging techniques and the application of artificial intelligence for biomarker discovery in chronic liver disease.

Data harmonisation for information fusion in digital healthcare: A state-of-the-art systematic review, meta-analysis and future research directions

Removing the bias and variance of multicentre data has always been a challenge in large scale digital healthcare studies, which requires the ability to integrate clinical features extracted from data acquired by different scanners and protocols to improve stability and robustness. Previous studies have described various computational approaches to fuse single modality multicentre datasets. However, these surveys rarely focused on evaluation metrics and lacked a checklist for computational data harmonisation studies. In this systematic review, we summarise the computational data harmonisation approaches for multi-modality data in the digital healthcare field, including harmonisation strategies and evaluation metrics based on different theories. In addition, a comprehensive checklist that summarises common practices for data harmonisation studies is proposed to guide researchers to report their research findings more effectively. Last but not least, flowcharts presenting possible ways for methodology and metric selection are proposed and the limitations of different methods have been surveyed for future research.

Automated brain tumour identification using magnetic resonance imaging: a systematic review and meta-analysis

Background

Automated brain tumor identification facilitates diagnosis and treatment planning. We evaluate the performance of traditional machine learning (TML) and deep learning (DL) in brain tumor detection and segmentation, using MRI.

Methods

A systematic literature search from January 2000 to May 8, 2021 was conducted. Study quality was assessed using the Checklist for Artificial Intelligence in Medical Imaging (CLAIM). Detection meta-analysis was performed using a unified hierarchical model. Segmentation studies were evaluated using a random effects model. Sensitivity analysis was performed for externally validated studies.

Results

Of 224 studies included in the systematic review, 46 segmentation and 38 detection studies were eligible for meta-analysis. In detection, DL achieved a lower false positive rate compared to TML; 0.018 (95% CI, 0.011 to 0.028) and 0.048 (0.032 to 0.072) (P < .001), respectively. In segmentation, DL had a higher dice similarity coefficient (DSC), particularly for tumor core (TC); 0.80 (0.77 to 0.83) and 0.63 (0.56 to 0.71) (P < .001), persisting on sensitivity analysis. Both manual and automated whole tumor (WT) segmentation had “good” (DSC ≥ 0.70) performance. Manual TC segmentation was superior to automated; 0.78 (0.69 to 0.86) and 0.64 (0.53 to 0.74) (P = .014), respectively. Only 30% of studies reported external validation.

Conclusions

The comparable performance of automated to manual WT segmentation supports its integration into clinical practice. However, manual outperformance for sub-compartmental segmentation highlights the need for further development of automated methods in this area. Compared to TML, DL provided superior performance for detection and sub-compartmental segmentation. Improvements in the quality and design of studies, including external validation, are required for the interpretability and generalizability of automated models.

Machine Learning for the Detection and Segmentation of Benign Tumors of the Central Nervous System: A Systematic Review

Simple Summary

Machine learning in radiology of the central nervous system has seen many interesting publications in the past few years. Since the focus has largely been on malignant tumors such as brain metastases and high-grade gliomas, we conducted a systematic review on benign tumors to summarize what has been published and where there might be gaps in the research. We found several studies that report good results, but the descriptions of methodologies could be improved to enable better comparisons and assessment of biases.

Abstract

Objectives: To summarize the available literature on using machine learning (ML) for the detection and segmentation of benign tumors of the central nervous system (CNS) and to assess the adherence of published ML/diagnostic accuracy studies to best practice. Methods: The MEDLINE database was searched for the use of ML in patients with any benign tumor of the CNS, and the records were screened according to PRISMA guidelines. Results: Eleven retrospective studies focusing on meningioma (n = 4), vestibular schwannoma (n = 4), pituitary adenoma (n = 2) and spinal schwannoma (n = 1) were included. The majority of studies attempted segmentation. Links to repositories containing code were provided in two manuscripts, and no manuscripts shared imaging data. Only one study used an external test set, which raises the question as to whether some of the good performances that have been reported were caused by overfitting and may not generalize to data from other institutions. Conclusions: Using ML for detecting and segmenting benign brain tumors is still in its infancy. Stronger adherence to ML best practices could facilitate easier comparisons between studies and contribute to the development of models that are more likely to one day be used in clinical practice.

Machine Learning Tools for Image-Based Glioma Grading and the Quality of Their Reporting: Challenges and Opportunities

Technological innovation has enabled the development of machine learning (ML) tools that aim to improve the practice of radiologists. In the last decade, ML applications to neuro-oncology have expanded significantly, with the pre-operative prediction of glioma grade using medical imaging as a specific area of interest. We introduce the subject of ML models for glioma grade prediction by remarking upon the models reported in the literature as well as by describing their characteristic developmental workflow and widely used classifier algorithms. The challenges facing these models-including data sources, external validation, and glioma grade classification methods -are highlighted. We also discuss the quality of how these models are reported, explore the present and future of reporting guidelines and risk of bias tools, and provide suggestions for the reporting of prospective works. Finally, this review offers insights into next steps that the field of ML glioma grade prediction can take to facilitate clinical implementation.

Meningioma Radiomics: At the Nexus of Imaging, Pathology and Biomolecular Characterization

Simple Summary

Meningiomas are typically benign, common extra-axial tumors of the central nervous system. Routine clinical assessment by radiologists presents some limitations regarding long-term patient outcome prediction and risk stratification. Given the exponential growth of interest in radiomics and artificial intelligence in medical imaging, numerous studies have evaluated the potential of these tools in the setting of meningioma imaging. These were aimed at the development of reliable and reproducible models based on quantitative data. Although several limitations have yet to be overcome for their routine use in clinical practice, their innovative potential is evident. In this review, we present a wide-ranging overview of radiomics and artificial intelligence applications in meningioma imaging.

Abstract

Meningiomas are the most common extra-axial tumors of the central nervous system (CNS). Even though recurrence is uncommon after surgery and most meningiomas are benign, an aggressive behavior may still be exhibited in some cases. Although the diagnosis can be made by radiologists, typically with magnetic resonance imaging, qualitative analysis has some limitations in regard to outcome prediction and risk stratification. The acquisition of this information could help the referring clinician in the decision-making process and selection of the appropriate treatment. Following the increased attention and potential of radiomics and artificial intelligence in the healthcare domain, including oncological imaging, researchers have investigated their use over the years to overcome the current limitations of imaging. The aim of these new tools is the replacement of subjective and, therefore, potentially variable medical image analysis by more objective quantitative data, using computational algorithms. Although radiomics has not yet fully entered clinical practice, its potential for the detection, diagnostic, and prognostic characterization of tumors is evident. In this review, we present a wide-ranging overview of radiomics and artificial intelligence applications in meningioma imaging.

Segmentation of Dental Restorations on Panoramic Radiographs Using Deep Learning

Convolutional Neural Networks (CNNs) such as U-Net have been widely used for medical image segmentation. Dental restorations are prominent features of dental radiographs. Applying U-Net on the panoramic image is challenging, as the shape, size and frequency of different restoration types vary. We hypothesized that models trained on smaller, equally spaced rectangular image crops (tiles) of the panoramic would outperform models trained on the full image. A total of 1781 panoramic radiographs were annotated pixelwise for fillings, crowns, and root canal fillings by dental experts. We used different numbers of tiles for our experiments. Five-times-repeated three-fold cross-validation was used for model evaluation. Training with more tiles improved model performance and accelerated convergence. The F1-score for the full panoramic image was 0.7, compared to 0.83, 0.92 and 0.95 for 6, 10 and 20 tiles, respectively. For root canals fillings, which are small, cone-shaped features that appear less frequently on the radiographs, the performance improvement was even higher (+294%). Training on tiles and pooling the results thereafter improved pixelwise classification performance and reduced the time to model convergence for segmenting dental restorations. Segmentation of panoramic radiographs is biased towards more frequent and extended classes. Tiling may help to overcome this bias and increase accuracy.

Validation of a machine learning software tool for automated large vessel occlusion detection in patients with suspected acute stroke

PURPOSE

CT angiography (CTA) is the imaging standard for large vessel occlusion (LVO) detection in patients with acute ischemic stroke. StrokeSENS LVO is an automated tool that utilizes a machine learning algorithm to identify anterior large vessel occlusions (LVO) on CTA. The aim of this study was to test the algorithm's performance in LVO detection in an independent dataset.

METHODS

A total of 400 studies (217 LVO, 183 other/no occlusion) read by expert consensus were used for retrospective analysis. The LVO was defined as intracranial internal carotid artery (ICA) occlusion and M1 middle cerebral artery (MCA) occlusion. Software performance in detecting anterior LVO was evaluated using receiver operator characteristics (ROC) analysis, reporting area under the curve (AUC), sensitivity, and specificity. Subgroup analyses were performed to evaluate if performance in detecting LVO differed by subgroups, namely M1 MCA and ICA occlusion sites, and in data stratified by patient age, sex, and CTA acquisition characteristics (slice thickness, kilovoltage tube peak, and scanner manufacturer).

RESULTS

AUC, sensitivity, and specificity overall were as follows: 0.939, 0.894, and 0.874, respectively, in the full cohort; 0.927, 0.857, and 0.874, respectively, in the ICA occlusion cohort; 0.945, 0.914, and 0.874, respectively, in the M1 MCA occlusion cohort. Performance did not differ significantly by patient age, sex, or CTA acquisition characteristics.

CONCLUSION

The StrokeSENS LVO machine learning algorithm detects anterior LVO with high accuracy from a range of scans in a large dataset.

Machine learning imaging applications in the differentiation of true tumour progression from treatment‐related effects in brain tumours: A systematic review and meta‐analysis

Introduction

Chemotherapy and radiotherapy can produce treatment‐related effects, which may mimic tumour progression. Advances in Artificial Intelligence (AI) offer the potential to provide a more consistent approach of diagnosis with improved accuracy. The aim of this study was to determine the efficacy of machine learning models to differentiate treatment‐related effects (TRE), consisting of pseudoprogression (PsP) and radiation necrosis (RN), and true tumour progression (TTP).

Methods

The systematic review was conducted in accordance with PRISMA‐DTA guidelines. Searches were performed on PubMed, Scopus, Embase, Medline (Ovid) and ProQuest databases. Quality was assessed according to the PROBAST and CLAIM criteria. There were 25 original full‐text journal articles eligible for inclusion.

Results

For gliomas: PsP versus TTP (16 studies, highest AUC = 0.98), RN versus TTP (4 studies, highest AUC = 0.9988) and TRE versus TTP (3 studies, highest AUC = 0.94). For metastasis: RN vs. TTP (2 studies, highest AUC = 0.81). A meta‐analysis was performed on 9 studies in the gliomas PsP versus TTP group using STATA. The meta‐analysis reported a high sensitivity of 95.2% (95%CI: 86.6–98.4%) and specificity of 82.4% (95%CI: 67.0–91.6%).

Conclusion

TRE can be distinguished from TTP with good performance using machine learning‐based imaging models. There remain issues with the quality of articles and the integration of models into clinical practice. Future studies should focus on the external validation of models and utilize standardized criteria such as CLAIM to allow for consistency in reporting.

DeepLN: A Multi-Task AI Tool to Predict the Imaging Characteristics, Malignancy and Pathological Subtypes in CT-Detected Pulmonary Nodules

Objectives

Distinction of malignant pulmonary nodules from the benign ones based on computed tomography (CT) images can be time-consuming but significant in routine clinical management. The advent of artificial intelligence (AI) has provided an opportunity to improve the accuracy of cancer risk prediction.

Methods

A total of 8950 detected pulmonary nodules with complete pathological results were retrospectively enrolled. The different radiological manifestations were identified mainly as various nodules densities and morphological features. Then, these nodules were classified into benign and malignant groups, both of which were subdivided into finer specific pathological types. Here, we proposed a deep convolutional neural network for the assessment of lung nodules named DeepLN to identify the radiological features and predict the pathologic subtypes of pulmonary nodules.

Results

In terms of density, the area under the receiver operating characteristic curves (AUCs) of DeepLN were 0.9707 (95% confidence interval, CI: 0.9645-0.9765), 0.7789 (95%CI: 0.7569-0.7995), and 0.8950 (95%CI: 0.8822-0.9088) for the pure-ground glass opacity (pGGO), mixed-ground glass opacity (mGGO) and solid nodules. As for the morphological features, the AUCs were 0.8347 (95%CI: 0.8193-0.8499) and 0.9074 (95%CI: 0.8834-0.9314) for spiculation and lung cavity respectively. For the identification of malignant nodules, our DeepLN algorithm achieved an AUC of 0.8503 (95%CI: 0.8319-0.8681) in the test set. Pertaining to predicting the pathological subtypes in the test set, the multi-task AUCs were 0.8841 (95%CI: 0.8567-0.9083) for benign tumors, 0.8265 (95%CI: 0.8004-0.8499) for inflammation, and 0.8022 (95%CI: 0.7616-0.8445) for other benign ones, while AUCs were 0.8675 (95%CI: 0.8525-0.8813) for lung adenocarcinoma (LUAD), 0.8792 (95%CI: 0.8640-0.8950) for squamous cell carcinoma (LUSC), 0.7404 (95%CI: 0.7031-0.7782) for other malignant ones respectively in the malignant group.

Conclusions

The DeepLN based on deep learning algorithm represented a competitive performance to predict the imaging characteristics, malignancy and pathologic subtypes on the basis of non-invasive CT images, and thus had great possibility to be utilized in the routine clinical workflow.

An endorectal ultrasound-based radiomics signature for preoperative prediction of lymphovascular invasion of rectal cancer

Objective

To investigate whether radiomics based on ultrasound images can predict lymphovascular invasion (LVI) of rectal cancer (RC) before surgery.

Methods

A total of 203 patients with RC were enrolled retrospectively, and they were divided into a training set (143 patients) and a validation set (60 patients). We extracted the radiomic features from the largest gray ultrasound image of the RC lesion. The intraclass correlation coefficient (ICC) was applied to test the repeatability of the radiomic features. The least absolute shrinkage and selection operator (LASSO) was used to reduce the data dimension and select significant features. Logistic regression (LR) analysis was applied to establish the radiomics model. The receiver operating characteristic (ROC) curve, calibration curve, and decision curve analysis (DCA) were used to evaluate the comprehensive performance of the model.

Results

Among the 203 patients, 33 (16.7%) were LVI positive and 170 (83.7%) were LVI negative. A total of 5350 (90.1%) radiomic features with ICC values of ≥ 0.75 were reported, which were subsequently subjected to hypothesis testing and LASSO regression dimension reduction analysis. Finally, 15 selected features were used to construct the radiomics model. The area under the curve (AUC) of the training set was 0.849, and the AUC of the validation set was 0.781. The calibration curve indicated that the radiomics model had good calibration, and DCA demonstrated that the model had clinical benefits.

Conclusion

The proposed endorectal ultrasound-based radiomics model has the potential to predict LVI preoperatively in RC.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12880-022-00813-6.

External Validation of Deep Learning Algorithms for Radiologic Diagnosis: A Systematic Review

Purpose

To assess generalizability of published deep learning (DL) algorithms for radiologic diagnosis.

Materials and Methods

In this systematic review, the PubMed database was searched for peer-reviewed studies of DL algorithms for image-based radiologic diagnosis that included external validation, published from January 1, 2015, through April 1, 2021. Studies using nonimaging features or incorporating non-DL methods for feature extraction or classification were excluded. Two reviewers independently evaluated studies for inclusion, and any discrepancies were resolved by consensus. Internal and external performance measures and pertinent study characteristics were extracted, and relationships among these data were examined using nonparametric statistics.

Results

Eighty-three studies reporting 86 algorithms were included. The vast majority (70 of 86, 81%) reported at least some decrease in external performance compared with internal performance, with nearly half (42 of 86, 49%) reporting at least a modest decrease (≥0.05 on the unit scale) and nearly a quarter (21 of 86, 24%) reporting a substantial decrease (≥0.10 on the unit scale). No study characteristics were found to be associated with the difference between internal and external performance.

Conclusion

Among published external validation studies of DL algorithms for image-based radiologic diagnosis, the vast majority demonstrated diminished algorithm performance on the external dataset, with some reporting a substantial performance decrease.Keywords: Meta-Analysis, Computer Applications-Detection/Diagnosis, Neural Networks, Computer Applications-General (Informatics), Epidemiology, Technology Assessment, Diagnosis, Informatics Supplemental material is available for this article. © RSNA, 2022.

LungNet22: A Fine-Tuned Model for Multiclass Classification and Prediction of Lung Disease Using X-ray Images

In recent years, lung disease has increased manyfold, causing millions of casualties annually. To combat the crisis, an efficient, reliable, and affordable lung disease diagnosis technique has become indispensable. In this study, a multiclass classification of lung disease from frontal chest X-ray imaging using a fine-tuned CNN model is proposed. The classification is conducted on 10 disease classes of the lungs, namely COVID-19, Effusion, Tuberculosis, Pneumonia, Lung Opacity, Mass, Nodule, Pneumothorax, and Pulmonary Fibrosis, along with the Normal class. The dataset is a collective dataset gathered from multiple sources. After pre-processing and balancing the dataset with eight augmentation techniques, a total of 80,000 X-ray images were fed to the model for classification purposes. Initially, eight pre-trained CNN models, AlexNet, GoogLeNet, InceptionV3, MobileNetV2, VGG16, ResNet 50, DenseNet121, and EfficientNetB7, were employed on the dataset. Among these, the VGG16 achieved the highest accuracy at 92.95%. To further improve the classification accuracy, LungNet22 was constructed upon the primary structure of the VGG16 model. An ablation study was used in the work to determine the different hyper-parameters. Using the Adam Optimizer, the proposed model achieved a commendable accuracy of 98.89%. To verify the performance of the model, several performance matrices, including the ROC curve and the AUC values, were computed as well.

The impact of radiomics for human papillomavirus status prediction in oropharyngeal cancer: systematic review and radiomics quality score assessment

Purpose

Human papillomavirus (HPV) status assessment is crucial for decision making in oropharyngeal cancer patients. In last years, several articles have been published investigating the possible role of radiomics in distinguishing HPV-positive from HPV-negative neoplasms. Aim of this review was to perform a systematic quality assessment of radiomic studies published on this topic.

Methods

Radiomics studies on HPV status prediction in oropharyngeal cancer patients were selected. The Radiomic Quality Score (RQS) was assessed by three readers to evaluate their methodological quality. In addition, possible correlations between RQS% and journal type, year of publication, impact factor, and journal rank were investigated.

Results

After the literature search, 19 articles were selected whose RQS median was 33% (range 0–42%). Overall, 16/19 studies included a well-documented imaging protocol, 13/19 demonstrated phenotypic differences, and all were compared with the current gold standard. No study included a public protocol, phantom study, or imaging at multiple time points. More than half (13/19) included feature selection and only 2 were comprehensive of non-radiomic features. Mean RQS was significantly higher in clinical journals.

Conclusion

Radiomics has been proposed for oropharyngeal cancer HPV status assessment, with promising results. However, these are supported by low methodological quality investigations. Further studies with higher methodological quality, appropriate standardization, and greater attention to validation are necessary prior to clinical adoption.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00234-022-02959-0.

Machine Learning Models for Classifying High- and Low-Grade Gliomas: A Systematic Review and Quality of Reporting Analysis

Objectives

To systematically review, assess the reporting quality of, and discuss improvement opportunities for studies describing machine learning (ML) models for glioma grade prediction.

Methods

This study followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses of Diagnostic Test Accuracy (PRISMA-DTA) statement. A systematic search was performed in September 2020, and repeated in January 2021, on four databases: Embase, Medline, CENTRAL, and Web of Science Core Collection. Publications were screened in Covidence, and reporting quality was measured against the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) Statement. Descriptive statistics were calculated using GraphPad Prism 9.

Results

The search identified 11,727 candidate articles with 1,135 articles undergoing full text review and 85 included in analysis. 67 (79%) articles were published between 2018-2021. The mean prediction accuracy of the best performing model in each study was 0.89 ± 0.09. The most common algorithm for conventional machine learning studies was Support Vector Machine (mean accuracy: 0.90 ± 0.07) and for deep learning studies was Convolutional Neural Network (mean accuracy: 0.91 ± 0.10). Only one study used both a large training dataset (n>200) and external validation (accuracy: 0.72) for their model. The mean adherence rate to TRIPOD was 44.5% ± 11.1%, with poor reporting adherence for model performance (0%), abstracts (0%), and titles (0%).

Conclusions

The application of ML to glioma grade prediction has grown substantially, with ML model studies reporting high predictive accuracies but lacking essential metrics and characteristics for assessing model performance. Several domains, including generalizability and reproducibility, warrant further attention to enable translation into clinical practice.

Systematic Review Registration

PROSPERO, identifier CRD42020209938.

Automated Detection, Segmentation, and Classification of Pericardial Effusions on Chest CT Using a Deep Convolutional Neural Network

Pericardial effusions (PEFs) are often missed on Computed Tomography (CT), which particularly affects the outcome of patients presenting with hemodynamic compromise. An automatic PEF detection, segmentation, and classification tool would expedite and improve CT based PEF diagnosis; 258 CTs with (206 with simple PEF, 52 with hemopericardium) and without PEF (each 134 with contrast, 124 non-enhanced) were identified using the radiology report (01/2016−01/2021). PEF were manually 3D-segmented. A deep convolutional neural network (nnU-Net) was trained on 316 cases and separately tested on the remaining 200 and 22 external post-mortem CTs. Inter-reader variability was tested on 40 CTs. PEF classification utilized the median Hounsfield unit from each prediction. The sensitivity and specificity for PEF detection was 97% (95% CI 91.48−99.38%) and 100.00% (95% CI 96.38−100.00%) and 89.74% and 83.61% for diagnosing hemopericardium (AUC 0.944, 95% CI 0.904−0.984). Model performance (Dice coefficient: 0.75 ± 0.01) was non-inferior to inter-reader (0.69 ± 0.02) and was unaffected by contrast administration nor alternative chest pathology (p > 0.05). External dataset testing yielded similar results. Our model reliably detects, segments, and classifies PEF on CT in a complex dataset, potentially serving as an alert tool whilst enhancing report quality. The model and corresponding datasets are publicly available.

Artificial Intelligence for COVID-19 Detection in Medical Imaging-Diagnostic Measures and Wasting-A Systematic Umbrella Review

The COVID-19 pandemic has sparked a barrage of primary research and reviews. We investigated the publishing process, time and resource wasting, and assessed the methodological quality of the reviews on artificial intelligence techniques to diagnose COVID-19 in medical images. We searched nine databases from inception until 1 September 2020. Two independent reviewers did all steps of identification, extraction, and methodological credibility assessment of records. Out of 725 records, 22 reviews analysing 165 primary studies met the inclusion criteria. This review covers 174,277 participants in total, including 19,170 diagnosed with COVID-19. The methodological credibility of all eligible studies was rated as critically low: 95% of papers had significant flaws in reporting quality. On average, 7.24 (range: 0-45) new papers were included in each subsequent review, and 14% of studies did not include any new paper into consideration. Almost three-quarters of the studies included less than 10% of available studies. More than half of the reviews did not comment on the previously published reviews at all. Much wasting time and resources could be avoided if referring to previous reviews and following methodological guidelines. Such information chaos is alarming. It is high time to draw conclusions from what we experienced and prepare for future pandemics.

Automated prediction of the neoadjuvant chemotherapy response in osteosarcoma with deep learning and an MRI-based radiomics nomogram

OBJECTIVES

To implement a pipeline to automatically segment the ROI and to use a nomogram integrating the MRI-based radiomics score and clinical variables to predict responses to neoadjuvant chemotherapy (NAC) in osteosarcoma patients.

METHODS

A total of 144 osteosarcoma patients treated with NAC were separated into training (n = 101) and test (n = 43) groups. After normalisation, ROIs for the preoperative MRI were segmented by a deep learning segmentation model trained with nnU-Net by using two independent manual segmentations as labels. Radiomics features were extracted using automatically segmented ROIs. Feature selection was performed in the training dataset by five-fold cross-validation. The clinical, radiomics, and clinical-radiomics models were built using multiple machine learning methods with the same training dataset and validated with the same test dataset. The segmentation model was evaluated by the Dice coefficient. AUC and decision curve analysis (DCA) were employed to illustrate the model performance and clinical utility.

RESULTS

36/144 (25.0%) patients were pathological good responders (pGRs) to NAC, while 108/144 (75.0%) were non-pGRs. The segmentation model achieved a Dice coefficient of 0.869 on the test dataset. The clinical and radiomics models reached AUCs of 0.636 with a 95% confidence interval (CI) of 0.427-0.860 and 0.759 (95% CI, 0.589-0.937), respectively, in the test dataset. The clinical-radiomics nomogram demonstrated good discrimination, with an AUC of 0.793 (95% CI, 0.610-0.975), and accuracy of 79.1%. The DCA suggested the clinical utility of the nomogram.

CONCLUSION

The automatic nomogram could be applied to aid radiologists in identifying pGRs to NAC.

KEY POINTS

• The nnU-Net trained by manual labels enables the use of an automatic segmentation tool for ROI delineation of osteosarcoma. • A pipeline using automatic lesion segmentation and followed by a radiomics classifier could aid the evaluation of NAC response of osteosarcoma. • A predictive nomogram composed of clinical variables and MRI-based radiomics score provides support for individualised treatment planning.

Machine Learning in Differentiating Gliomas from Primary CNS Lymphomas: A Systematic Review, Reporting Quality, and Risk of Bias Assessment

BACKGROUND

Differentiating gliomas and primary CNS lymphoma represents a diagnostic challenge with important therapeutic ramifications. Biopsy is the preferred method of diagnosis, while MR imaging in conjunction with machine learning has shown promising results in differentiating these tumors.

PURPOSE

Our aim was to evaluate the quality of reporting and risk of bias, assess data bases with which the machine learning classification algorithms were developed, the algorithms themselves, and their performance.

DATA SOURCES

Ovid EMBASE, Ovid MEDLINE, Cochrane Central Register of Controlled Trials, and the Web of Science Core Collection were searched according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines.

STUDY SELECTION

From 11,727 studies, 23 peer-reviewed studies used machine learning to differentiate primary CNS lymphoma from gliomas in 2276 patients.

DATA ANALYSIS

Characteristics of data sets and machine learning algorithms were extracted. A meta-analysis on a subset of studies was performed. Reporting quality and risk of bias were assessed using the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) and Prediction Model Study Risk Of Bias Assessment Tool.

DATA SYNTHESIS

The highest area under the receiver operating characteristic curve (0.961) and accuracy (91.2%) in external validation were achieved by logistic regression and support vector machines models using conventional radiomic features. Meta-analysis of machine learning classifiers using these features yielded a mean area under the receiver operating characteristic curve of 0.944 (95% CI, 0.898-0.99). The median TRIPOD score was 51.7%. The risk of bias was high for 16 studies.

LIMITATIONS

Exclusion of abstracts decreased the sensitivity in evaluating all published studies. Meta-analysis had high heterogeneity.

CONCLUSIONS

Machine learning-based methods of differentiating primary CNS lymphoma from gliomas have shown great potential, but most studies lack large, balanced data sets and external validation. Assessment of the studies identified multiple deficiencies in reporting quality and risk of bias. These factors reduce the generalizability and reproducibility of the findings.

Comparative performance of fully-automated and semi-automated artificial intelligence methods for the detection of clinically significant prostate cancer on MRI: a systematic review

OBJECTIVES

We systematically reviewed the current literature evaluating the ability of fully-automated deep learning (DL) and semi-automated traditional machine learning (TML) MRI-based artificial intelligence (AI) methods to differentiate clinically significant prostate cancer (csPCa) from indolent PCa (iPCa) and benign conditions.

METHODS

We performed a computerised bibliographic search of studies indexed in MEDLINE/PubMed, arXiv, medRxiv, and bioRxiv between 1 January 2016 and 31 July 2021. Two reviewers performed the title/abstract and full-text screening. The remaining papers were screened by four reviewers using the Checklist for Artificial Intelligence in Medical Imaging (CLAIM) for DL studies and Radiomics Quality Score (RQS) for TML studies. Papers that fulfilled the pre-defined screening requirements underwent full CLAIM/RQS evaluation alongside the risk of bias assessment using QUADAS-2, both conducted by the same four reviewers. Standard measures of discrimination were extracted for the developed predictive models.

RESULTS

17/28 papers (five DL and twelve TML) passed the quality screening and were subject to a full CLAIM/RQS/QUADAS-2 assessment, which revealed a substantial study heterogeneity that precluded us from performing quantitative analysis as part of this review. The mean RQS of TML papers was 11/36, and a total of five papers had a high risk of bias. AUCs of DL and TML papers with low risk of bias ranged between 0.80-0.89 and 0.75-0.88, respectively.

CONCLUSION

We observed comparable performance of the two classes of AI methods and identified a number of common methodological limitations and biases that future studies will need to address to ensure the generalisability of the developed models.

Deep learning for the classification of cervical maturation degree and pubertal growth spurts: A pilot study

Objective

This study aimed to present and evaluate a new deep learning model for determining cervical vertebral maturation (CVM) degree and growth spurts by analyzing lateral cephalometric radiographs.

Methods

The study sample included 890 cephalograms. The images were classified into six cervical stages independently by two orthodontists. The images were also categorized into three degrees on the basis of the growth spurt pre-pubertal, growth spurt, and post-pubertal. Subsequently, the samples were fed to a transfer learning model implemented using the Python programming language and PyTorch library. In the last step, the test set of cephalograms was randomly coded and provided to two new orthodontists in order to compare their diagnosis to the artificial intelligence (AI) model’s performance using weighted kappa and Cohen’s kappa statistical analyses.

Results

The model’s validation and test accuracy for the six-class CVM diagnosis were 62.63% and 61.62%, respectively. Moreover, the model’s validation and test accuracy for the three-class classification were 75.76% and 82.83%, respectively. Furthermore, substantial agreements were observed between the two orthodontists as well as one of them and the AI model.

Conclusions

The newly developed AI model had reasonable accuracy in detecting the CVM stage and high reliability in detecting the pubertal stage. However, its accuracy was still less than that of human observers. With further improvements in data quality, this model should be able to provide practical assistance to practicing dentists in the future.

Mask-less Two-dimensional Digital Subtraction Angiography Generation Model for Abdominal Vasculature using Deep Learning

PURPOSE

To develop a deep learning model to generate synthetic, two-dimensional subtraction angiography images free of artifacts from native abdominal angiograms.

MATERIALS AND METHODS

In this retrospective study, two-dimensional digital subtraction angiograms (2D-DSA) and native angiograms were consecutively collected from July 2019 to March 2020. Images were divided into motion-free (training, validation, and motion-free test datasets) and containing motion artifacts (motion-artifact test dataset) sets. A total of 3185, 393, 383, and 345 images from 87 patients (mean age, 71 ± 10 years; 64 men, 23 women) were included in the training, validation, motion-free, and motion-artifacts test datasets, respectively. Native angiograms and 2D-DSA image pairs were used to train and validate an image-to-image translation model to generate synthetic deep learning-based subtraction angiography (DLSA) images. DLSA images were quantitatively evaluated by peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) using the motion-free dataset and were qualitatively evaluated by visual assessments by radiologists with a numerical rating scale using the motion-artifacts dataset.

RESULTS

The DLSA images showed mean PSNR (± standard deviation) of 43.05 ± 3.65 dB and mean SSIM of 0.98 ± 0.01, indicating high agreement with the original 2D-DSA images in the motion-free dataset. Qualitative visual evaluation by radiologists on the motion-artifacts dataset showed that DLSA images contained fewer motion artifacts than 2D-DSA. Additionally, DLSA images scored similarly to or higher than 2D-DSA images for vascular visualization and clinical usefulness.

CONCLUSION

The developed deep learning model could generate synthetic, motion-free subtraction images from abdominal angiograms with similar imaging characteristics to 2D-DSA images.

Machine Learning Applications for Differentiation of Glioma from Brain Metastasis-A Systematic Review

Simple Summary

We present a systematic review of published reports on machine learning (ML) applications for the differentiation of gliomas from brain metastases by summarizing study characteristics, strengths, and pitfalls. Based on these findings, we present recommendations for future research in this field.

Abstract

Glioma and brain metastasis can be difficult to distinguish on conventional magnetic resonance imaging (MRI) due to the similarity of imaging features in specific clinical circumstances. Multiple studies have investigated the use of machine learning (ML) models for non-invasive differentiation of glioma from brain metastasis. Many of the studies report promising classification results, however, to date, none have been implemented into clinical practice. After a screening of 12,470 studies, we included 29 eligible studies in our systematic review. From each study, we aggregated data on model design, development, and best classifiers, as well as quality of reporting according to the TRIPOD statement. In a subset of eligible studies, we conducted a meta-analysis of the reported AUC. It was found that data predominantly originated from single-center institutions (n = 25/29) and only two studies performed external validation. The median TRIPOD adherence was 0.48, indicating insufficient quality of reporting among surveyed studies. Our findings illustrate that despite promising classification results, reliable model assessment is limited by poor reporting of study design and lack of algorithm validation and generalizability. Therefore, adherence to quality guidelines and validation on outside datasets is critical for the clinical translation of ML for the differentiation of glioma and brain metastasis.

Reporting of Model Performance and Statistical Methods in Studies That Use Machine Learning to Develop Clinical Prediction Models: Protocol for a Systematic Review

BACKGROUND

With the growing excitement of the potential benefits of using machine learning and artificial intelligence in medicine, the number of published clinical prediction models that use these approaches has increased. However, there is evidence (albeit limited) that suggests that the reporting of machine learning-specific aspects in these studies is poor. Further, there are no reviews assessing the reporting quality or broadly accepted reporting guidelines for these aspects.

OBJECTIVE

This paper presents the protocol for a systematic review that will assess the reporting quality of machine learning-specific aspects in studies that use machine learning to develop clinical prediction models.

METHODS

We will include studies that use a supervised machine learning algorithm to develop a prediction model for use in clinical practice (ie, for diagnosis or prognosis of a condition or identification of candidates for health care interventions). We will search MEDLINE for studies published in 2019, pseudorandomly sort the records, and screen until we obtain 100 studies that meet our inclusion criteria. We will assess reporting quality with a novel checklist developed in parallel with this review, which includes content derived from existing reporting guidelines, textbooks, and consultations with experts. The checklist will cover 4 key areas where the reporting of machine learning studies is unique: modelling steps (order and data used for each step), model performance (eg, reporting the performance of each model compared), statistical methods (eg, describing the tuning approach), and presentation of models (eg, specifying the predictors that contributed to the final model).

RESULTS

We completed data analysis in August 2021 and are writing the manuscript. We expect to submit the results to a peer-reviewed journal in early 2022.

CONCLUSIONS

This review will contribute to more standardized and complete reporting in the field by identifying areas where reporting is poor and can be improved.

TRIAL REGISTRATION

PROSPERO International Prospective Register of Systematic Reviews CRD42020206167; https://www.crd.york.ac.uk/PROSPERO/display_record.php?RecordID=206167.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR1-10.2196/30956.

Automated COVID-19 diagnosis and prognosis with medical imaging and who is publishing: a systematic review

OBJECTIVES

 To conduct a systematic survey of published techniques for automated diagnosis and prognosis of COVID-19 diseases using medical imaging, assessing the validity of reported performance and investigating the proposed clinical use-case. To conduct a scoping review into the authors publishing such work.

METHODS

 The Scopus database was queried and studies were screened for article type, and minimum source normalized impact per paper and citations, before manual relevance assessment and a bias assessment derived from a subset of the Checklist for Artificial Intelligence in Medical Imaging (CLAIM). The number of failures of the full CLAIM was adopted as a surrogate for risk-of-bias. Methodological and performance measurements were collected from each technique. Each study was assessed by one author. Comparisons were evaluated for significance with a two-sided independent t-test.

FINDINGS

 Of 1002 studies identified, 390 remained after screening and 81 after relevance and bias exclusion. The ratio of exclusion for bias was 71%, indicative of a high level of bias in the field. The mean number of CLAIM failures per study was 8.3 ± 3.9 [1,17] (mean ± standard deviation [min,max]). 58% of methods performed diagnosis versus 31% prognosis. Of the diagnostic methods, 38% differentiated COVID-19 from healthy controls. For diagnostic techniques, area under the receiver operating curve (AUC) = 0.924 ± 0.074 [0.810,0.991] and accuracy = 91.7% ± 6.4 [79.0,99.0]. For prognostic techniques, AUC = 0.836 ± 0.126 [0.605,0.980] and accuracy = 78.4% ± 9.4 [62.5,98.0]. CLAIM failures did not correlate with performance, providing confidence that the highest results were not driven by biased papers. Deep learning techniques reported higher AUC (p < 0.05) and accuracy (p < 0.05), but no difference in CLAIM failures was identified.

INTERPRETATION

 A majority of papers focus on the less clinically impactful diagnosis task, contrasted with prognosis, with a significant portion performing a clinically unnecessary task of differentiating COVID-19 from healthy. Authors should consider the clinical scenario in which their work would be deployed when developing techniques. Nevertheless, studies report superb performance in a potentially impactful application. Future work is warranted in translating techniques into clinical tools.

Deployed Deep Learning Kidney Segmentation for Polycystic Kidney Disease MRI

This study develops, validates, and deploys deep learning for automated total kidney volume (TKV) measurement (a marker of disease severity) on T2-weighted MRI studies of autosomal dominant polycystic kidney disease (ADPKD). The model was based on the U-Net architecture with an EfficientNet encoder, developed using 213 abdominal MRI studies in 129 patients with ADPKD. Patients were randomly divided into 70% training, 15% validation, and 15% test sets for model development. Model performance was assessed using Dice similarity coefficient (DSC) and Bland-Altman analysis. External validation in 20 patients from outside institutions demonstrated a DSC of 0.98 (IQR, 0.97-0.99) and a Bland-Altman difference of 2.6% (95% CI: 1.0%, 4.1%). Prospective validation in 53 patients demonstrated a DSC of 0.97 (IQR, 0.94-0.98) and a Bland-Altman difference of 3.6% (95% CI: 2.0%, 5.2%). Last, the efficiency of model-assisted annotation was evaluated on the first 50% of prospective cases (n = 28), with a 51% mean reduction in contouring time (P < .001), from 1724 seconds (95% CI: 1373, 2075) to 723 seconds (95% CI: 555, 892). In conclusion, our deployed artificial intelligence pipeline accurately performs automated segmentation for TKV estimation of polycystic kidneys and reduces expert contouring time. Keywords: Convolutional Neural Network (CNN), Segmentation, Kidney ClinicalTrials.gov identification no.: NCT00792155 Supplemental material is available for this article. © RSNA, 2022.

Assessing Methods and Tools to Improve Reporting, Increase Transparency, and Reduce Failures in Machine Learning Applications in Health Care

Artificial intelligence applications for health care have come a long way. Despite the remarkable progress, there are several examples of unfulfilled promises and outright failures. There is still a struggle to translate successful research into successful real-world applications. Machine learning (ML) products diverge from traditional software products in fundamental ways. Particularly, the main component of an ML solution is not a specific piece of code that is written for a specific purpose; rather, it is a generic piece of code, a model, customized by a training process driven by hyperparameters and a dataset. Datasets are usually large, and models are opaque. Therefore, datasets and models cannot be inspected in the same, direct way as traditional software products. Other methods are needed to detect failures in ML products. This report investigates recent advancements that promote auditing, supported by transparency, as a mechanism to detect potential failures in ML products for health care applications. It reviews practices that apply to the early stages of the ML lifecycle, when datasets and models are created; these stages are unique to ML products. Concretely, this report demonstrates how two recently proposed checklists, datasheets for datasets and model cards, can be adopted to increase the transparency of crucial stages of the ML lifecycle, using ChestX-ray8 and CheXNet as examples. The adoption of checklists to document the strengths, limitations, and applications of datasets and models in a structured format leads to increased transparency, allowing early detection of potential problems and opportunities for improvement. Keywords: Artificial Intelligence, Machine Learning, Lifecycle, Auditing, Transparency, Failures, Datasheets, Datasets, Model Cards Supplemental material is available for this article. © RSNA, 2022.

Effective deep learning approaches for predicting COVID-19 outcomes from chest computed tomography volumes

The rapid evolution of the novel coronavirus disease (COVID-19) pandemic has resulted in an urgent need for effective clinical tools to reduce transmission and manage severe illness. Numerous teams are quickly developing artificial intelligence approaches to these problems, including using deep learning to predict COVID-19 diagnosis and prognosis from chest computed tomography (CT) imaging data. In this work, we assess the value of aggregated chest CT data for COVID-19 prognosis compared to clinical metadata alone. We develop a novel patient-level algorithm to aggregate the chest CT volume into a 2D representation that can be easily integrated with clinical metadata to distinguish COVID-19 pneumonia from chest CT volumes from healthy participants and participants with other viral pneumonia. Furthermore, we present a multitask model for joint segmentation of different classes of pulmonary lesions present in COVID-19 infected lungs that can outperform individual segmentation models for each task. We directly compare this multitask segmentation approach to combining feature-agnostic volumetric CT classification feature maps with clinical metadata for predicting mortality. We show that the combination of features derived from the chest CT volumes improve the AUC performance to 0.80 from the 0.52 obtained by using patients' clinical data alone. These approaches enable the automated extraction of clinically relevant features from chest CT volumes for risk stratification of COVID-19 patients.

Artificial intelligence in liver diseases: improving diagnostics, prognostics and response prediction

Clinical routine in hepatology involves the diagnosis and treatment of a wide spectrum of metabolic, infectious, autoimmune and neoplastic diseases. Clinicians integrate qualitative and quantitative information from multiple data sources to make a diagnosis, prognosticate the disease course, and recommend a treatment. In the last 5 years, advances in artificial intelligence (AI), particularly in deep learning, have made it possible to extract clinically relevant information from complex and diverse clinical datasets. In particular, histopathology and radiology image data contain diagnostic, prognostic and predictive information which AI can extract. Ultimately, such AI systems could be implemented in clinical routine as decision support tools. However, in the context of hepatology, this requires further large-scale clinical validation and regulatory approval. Herein, we summarise the state of the art in AI in hepatology with a particular focus on histopathology and radiology data. We present a roadmap for the further development of novel biomarkers in hepatology and outline critical obstacles which need to be overcome.

Deep learning for automated detection and numbering of permanent teeth on panoramic images

OBJECTIVE

This study aimed to evaluate an automated detection system to detect and classify permanent teeth on orthopantomogram (OPG) images using convolutional neural networks (CNNs).

METHODS

In total, 591 digital OPGs were collected from patients older than 18 years. Three qualified dentists performed individual teeth labelling on images to generate the ground truth annotations. A three-step procedure, relying upon CNNs, was proposed for automated detection and classification of teeth. Firstly, U-Net, a type of CNN, performed preliminary segmentation of tooth regions or detecting regions of interest (ROIs) on panoramic images. Secondly, the Faster R-CNN, an advanced object detection architecture, identified each tooth within the ROI determined by the U-Net. Thirdly, VGG-16 architecture classified each tooth into 32 categories, and a tooth number was assigned. A total of 17,135 teeth cropped from 591 radiographs were used to train and validate the tooth detection and tooth numbering modules. 90% of OPG images were used for training, and the remaining 10% were used for validation. 10-folds cross-validation was performed for measuring the performance. The intersection over union (IoU), F1 score, precision, and recall (i.e. sensitivity) were used as metrics to evaluate the performance of resultant CNNs.

RESULTS

The ROI detection module had an IoU of 0.70. The tooth detection module achieved a recall of 0.99 and a precision of 0.99. The tooth numbering module had a recall, precision and F1 score of 0.98.

CONCLUSION

The resultant automated method achieved high performance for automated tooth detection and numbering from OPG images. Deep learning can be helpful in the automatic filing of dental charts in general dentistry and forensic medicine.

Critical Analysis of Artificial Intelligence in Endodontics: A Scoping Review

INTRODUCTION

Artificial intelligence (AI) comprises computational models that mimic the human brain to perform various diagnostic tasks in clinical practice. The aim of this scoping review was to systematically analyze the AI algorithms and models used in endodontics and identify the source quality and type of evidence.

METHODS

A literature search was conducted in October 2020 to identify the relevant literature in English language in the 4 major health sciences databases, ie, MEDLINE, Dentistry & Oral Science, CINAHL Plus, and Cochrane Library. Our review questions were the following: what are the different AI algorithms and models used in endodontics?, what are the datasets being used?, what type of performance metrics were reported?, and what diagnostic performance measures were used?. The quality of the included studies was evaluated by a modified Quality Assessment of Studies of Diagnostic Accuracy risk (QUADAS) tool.

RESULTS

Out of 300 studies, 12 articles met our inclusion criteria and were subjected to final analysis. Among the included studies, 6 studies focused on periapical pathology, and 3 studies investigated vertical root fractures. Most studies (n = 10) used neural networks, among which convolutional neural networks were commonly used. The datasets that were mostly studied were radiographs. Out of 12 studies, only 3 studies achieved a high score according to the modified QUADAS tool.

CONCLUSIONS

AI models had acceptable performance, ie, accuracy >90% in executing various diagnostic tasks. The scientific reporting of AI-related research is irregular. The endodontic community needs to implement recommended guidelines to improve the weaknesses in the current planning and reporting of AI-related research to improve its scientific vigor.

Artificial Intelligence for Interstitial Lung Disease Analysis on Chest Computed Tomography: A Systematic Review

RATIONALE AND OBJECTIVES

High-resolution computed tomography (HRCT) is paramount in the assessment of interstitial lung disease (ILD). Yet, HRCT interpretation of ILDs may be hampered by inter- and intra-observer variability. Recently, artificial intelligence (AI) has revolutionized medical image analysis. This technology has the potential to advance patient care in ILD. We aimed to systematically evaluate the application of AI for the analysis of ILD in HRCT.

MATERIALS AND METHODS

We searched MEDLINE/PubMed databases for original publications of deep learning for ILD analysis on chest CT. The search included studies published up to March 1, 2021. The risk of bias evaluation included tailored Quality Assessment of Diagnostic Accuracy Studies and the modified Joanna Briggs Institute Critical Appraisal checklist.

RESULTS

Data was extracted from 19 retrospective studies. Deep learning techniques included detection, segmentation, and classification of ILD on HRCT. Most studies focused on the classification of ILD into different morphological patterns. Accuracies of 78%-91% were achieved. Two studies demonstrated near-expert performance for the diagnosis of idiopathic pulmonary fibrosis (IPF). The Quality Assessment of Diagnostic Accuracy Studies tool identified a high risk of bias in 15/19 (78.9%) of the studies.

CONCLUSION

AI has the potential to contribute to the radiologic diagnosis and classification of ILD. However, the accuracy performance is still not satisfactory, and research is limited by a small number of retrospective studies. Hence, the existing published data may not be sufficiently reliable. Only well-designed prospective controlled studies can accurately assess the value of existing AI tools for ILD evaluation.

Machine intelligence in non-invasive endocrine cancer diagnostics

Artificial intelligence (AI) has illuminated a clear path towards an evolving health-care system replete with enhanced precision and computing capabilities. Medical imaging analysis can be strengthened by machine learning as the multidimensional data generated by imaging naturally lends itself to hierarchical classification. In this Review, we describe the role of machine intelligence in image-based endocrine cancer diagnostics. We first provide a brief overview of AI and consider its intuitive incorporation into the clinical workflow. We then discuss how AI can be applied for the characterization of adrenal, pancreatic, pituitary and thyroid masses in order to support clinicians in their diagnostic interpretations. This Review also puts forth a number of key evaluation criteria for machine learning in medicine that physicians can use in their appraisals of these algorithms. We identify mitigation strategies to address ongoing challenges around data availability and model interpretability in the context of endocrine cancer diagnosis. Finally, we delve into frontiers in systems integration for AI, discussing automated pipelines and evolving computing platforms that leverage distributed, decentralized and quantum techniques.

Imaging Biomarkers of Glioblastoma Treatment Response: A Systematic Review and Meta-Analysis of Recent Machine Learning Studies

OBJECTIVE

Monitoring biomarkers using machine learning (ML) may determine glioblastoma treatment response. We systematically reviewed quality and performance accuracy of recently published studies.

METHODS

Following Preferred Reporting Items for Systematic Reviews and Meta-Analysis: Diagnostic Test Accuracy, we extracted articles from MEDLINE, EMBASE and Cochrane Register between 09/2018-01/2021. Included study participants were adults with glioblastoma having undergone standard treatment (maximal resection, radiotherapy with concomitant and adjuvant temozolomide), and follow-up imaging to determine treatment response status (specifically, distinguishing progression/recurrence from progression/recurrence mimics, the target condition). Using Quality Assessment of Diagnostic Accuracy Studies Two/Checklist for Artificial Intelligence in Medical Imaging, we assessed bias risk and applicability concerns. We determined test set performance accuracy (sensitivity, specificity, precision, F1-score, balanced accuracy). We used a bivariate random-effect model to determine pooled sensitivity, specificity, area-under the receiver operator characteristic curve (ROC-AUC). Pooled measures of balanced accuracy, positive/negative likelihood ratios (PLR/NLR) and diagnostic odds ratio (DOR) were calculated. PROSPERO registered (CRD42021261965).

RESULTS

Eighteen studies were included (1335/384 patients for training/testing respectively). Small patient numbers, high bias risk, applicability concerns (particularly confounding in reference standard and patient selection) and low level of evidence, allow limited conclusions from studies. Ten studies (10/18, 56%) included in meta-analysis gave 0.769 (0.649-0.858) sensitivity [pooled (95% CI)]; 0.648 (0.749-0.532) specificity; 0.706 (0.623-0.779) balanced accuracy; 2.220 (1.560-3.140) PLR; 0.366 (0.213-0.572) NLR; 6.670 (2.800-13.500) DOR; 0.765 ROC-AUC.

CONCLUSION

ML models using MRI features to distinguish between progression and mimics appear to demonstrate good diagnostic performance. However, study quality and design require improvement.

MRI-based radiomics analysis improves preoperative diagnostic performance for the depth of stromal invasion in patients with early stage cervical cancer

Background

The depth of cervical stromal invasion is one of the important prognostic factors affecting decision-making for early stage cervical cancer (CC). This study aimed to develop and validate a T2-weighted imaging (T2WI)-based radiomics model and explore independent risk factors (factors with statistical significance in both univariate and multivariate analyses) of middle or deep stromal invasion in early stage CC.

Methods

Between March 2017 and March 2021, a total of 234 International Federation of Gynecology and Obstetrics IB1-IIA1 CC patients were enrolled and randomly divided into a training cohort (n = 188) and a validation cohort (n = 46). The radiomics features of each patient were extracted from preoperative sagittal T2WI, and key features were selected. After independent risk factors were identified, a combined model and nomogram incorporating radiomics signature and independent risk factors were developed. Diagnostic accuracy of radiologists was also evaluated.

Results

The maximal tumor diameter (MTD) on magnetic resonance imaging was identified as an independent risk factor. In the validation cohort, the radiomics model, MTD, and combined model showed areas under the curve of 0.879, 0.844, and 0.886. The radiomics model and combined model showed the same sensitivity and specificity of 87.9% and 84.6%, which were better than radiologists (sensitivity, senior = 75.7%, junior = 63.6%; specificity, senior = 69.2%, junior = 53.8%) and MTD (sensitivity = 69.7%, specificity = 76.9%).

Conclusion

MRI-based radiomics analysis outperformed radiologists for the preoperative diagnosis of middle or deep stromal invasion in early stage CC, and the probability can be individually evaluated by a nomogram.

Automated grading of enlarged perivascular spaces in clinical imaging data of an acute stroke cohort using an interpretable, 3D deep learning framework

Enlarged perivascular spaces (EPVS), specifically in stroke patients, has been shown to strongly correlate with other measures of small vessel disease and cognitive impairment at 1 year follow-up. Typical grading of EPVS is often challenging and time consuming and is usually based on a subjective visual rating scale. The purpose of the current study was to develop an interpretable, 3D neural network for grading enlarged perivascular spaces (EPVS) severity at the level of the basal ganglia using clinical-grade imaging in a heterogenous acute stroke cohort, in the context of total cerebral small vessel disease (CSVD) burden. T2-weighted images from a retrospective cohort of 262 acute stroke patients, collected in 2015 from 5 regional medical centers, were used for analyses. Patients were given a label of 0 for none-to-mild EPVS (< 10) and 1 for moderate-to-severe EPVS (≥ 10). A three-dimensional residual network of 152 layers (3D-ResNet-152) was created to predict EPVS severity and 3D gradient class activation mapping (3DGradCAM) was used for visual interpretation of results. Our model achieved an accuracy 0.897 and area-under-the-curve of 0.879 on a hold-out test set of 15% of the total cohort (n = 39). 3DGradCAM showed areas of focus that were in physiologically valid locations, including other prevalent areas for EPVS. These maps also suggested that distribution of class activation values is indicative of the confidence in the model's decision. Potential clinical implications of our results include: (1) support for feasibility of automated of EPVS scoring using clinical-grade neuroimaging data, potentially alleviating rater subjectivity and improving confidence of visual rating scales, and (2) demonstration that explainable models are critical for clinical translation.

Synthetic FLAIR as a Substitute for FLAIR Sequence in Acute Ischemic Stroke

Background In acute ischemic stroke (AIS), fluid-attenuated inversion recovery (FLAIR) is used for treatment decisions when onset time is unknown. Synthetic FLAIR could be generated with deep learning from information embedded in diffusion-weighted imaging (DWI) and could replace acquired FLAIR sequence (real FLAIR) and shorten MRI duration. Purpose To compare performance of synthetic and real FLAIR for DWI-FLAIR mismatch estimation and identification of patients presenting within 4.5 hours from symptom onset. Materials and Methods In this retrospective study, all pretreatment and early follow-up (<48 hours after symptom onset) MRI data sets including DWI (b = 0-1000 sec/mm²) and FLAIR sequences obtained in consecutive patients with AIS referred for reperfusion therapies between January 2002 and May 2019 were included. On the training set (80%), a generative adversarial network was trained to produce synthetic FLAIR with DWI as input. On the test set (20%), synthetic FLAIR was computed without real FLAIR knowledge. The DWI-FLAIR mismatch was evaluated on both FLAIR data sets by four independent readers. Interobserver reproducibility and DWI-FLAIR mismatch concordance between synthetic and real FLAIR were evaluated with κ statistics. Sensitivity and specificity for identification of AIS within 4.5 hours were compared in patients with known onset time by using McNemar test. Results The study included 1416 MRI scans (861 patients; median age, 71 years [interquartile range, 57-81 years]; 375 men), yielding 1134 and 282 scans for training and test sets, respectively. Regarding DWI-FLAIR mismatch, interobserver reproducibility was substantial for real and synthetic FLAIR (κ = 0.80 [95% CI: 0.74, 0.87] and 0.80 [95% CI: 0.74, 0.87], respectively). After consensus, concordance between real and synthetic FLAIR was almost perfect (κ = 0.88; 95% CI: 0.82, 0.93). Diagnostic value for identifying AIS within 4.5 hours did not differ between real and synthetic FLAIR (sensitivity: 107 of 131 [82%] vs 111 of 131 [85%], P = .2; specificity: 96 of 104 [92%] vs 96 of 104 [92%], respectively, P > .99). Conclusion Synthetic fluid-attenuated inversion recovery (FLAIR) had diagnostic performances similar to real FLAIR in depicting diffusion-weighted imaging-FLAIR mismatch and in helping to identify early acute ischemic stroke, and it may accelerate MRI protocols. © RSNA, 2022 Online supplemental material is available for this article. See also the editorial by Carroll and Hurley in this issue.