Overview of deep learning in medical imaging

Exploring the values underlying machine learning research in medical image analysis

Machine learning has emerged as a crucial tool for medical image analysis, largely due to recent developments in deep artificial neural networks addressing numerous, diverse clinical problems. As with any conceptual tool, the effective use of machine learning should be predicated on an understanding of its underlying motivations just as much as algorithms or theory - and to do so, we need to explore its philosophical foundations. One of these foundations is the understanding of how values, despite being non-empirical, nevertheless affect scientific research. This article has three goals: to introduce the reader to values in a way that is specific to medical image analysis; to characterise a particular set of technical decisions (what we call the end-to-end vs. separable learning spectrum) that are fundamental to machine learning for medical image analysis; and to create a simple and structured method to show how these values can be rigorously connected to these technical decisions. This better understanding of how the philosophy of science can clarify fundamental elements of how medical image analysis research is performed and can be improved.

How New Technologies Will Transform Total Knee Arthroplasty from a Singular Surgical Procedure to a Holistic Standardized Process

Many new technologies focused mainly on improving surgical accuracy were first developed in total knee arthroplasty and have not yet shown significant value. These non-significant clinical improvements could potentially be explained by an inadequate target. In this current concept paper, the authors will describe how artificial intelligence (AI), robotic-assisted surgery, and custom implants allow the definition of new targets and the standardization of the TKA process. As paradoxical as it may be, new technologies in TKA will allow for better standardization in the overall way in which this procedure is carried out. Achieving this goal can be accomplished by incorporating AI-driven tools into the medical field. These tools are intended to enhance decision making, refine surgical planning, and increase the precision and consistency of surgical procedures. Moreover, custom implants with personalized alignment, beyond restoring native anatomy, will define new targets and standardize the whole TKA process.

Artificial intelligence based vision transformer application for grading histopathological images of oral epithelial dysplasia: a step towards AI-driven diagnosis

BACKGROUND

This study aimed to classify dysplastic and healthy oral epithelial histopathological images, according to WHO and binary grading systems, using the Vision Transformer (ViT) deep learning algorithm-a state-of-the-art Artificial Intelligence (AI) approach and compare it with established Convolutional Neural Network models (VGG16 and ConvNet).

METHODS

A total of 218 histopathological slide images were collected from the Department of Oral and Maxillofacial Pathology at Tehran University of Medical Sciences archive and combined with two online databases. Two oral pathologists independently labeled the images based on the 2022 World Health Organization (WHO) grading system (mild, moderate and severe), the binary grading system (low risk and high risk), including an additional normal tissue class. After preprocessing, the images were fed to the ViT, VGG16 and ConvNet models.

RESULTS

Image preprocessing yielded 2,545 low-risk, 2,054 high-risk, 726 mild, 831 moderate, 449 severe, and 937 normal tissue patches. The proposed ViT model outperformed both CNNs with the accuracy of 94% (VGG16:86% and ConvNet: 88%) in 3-class scenario and 97% (VGG16:79% and ConvNet: 88%) in 4-class scenario.

CONCLUSIONS

The ViT model successfully classified oral epithelial dysplastic tissues with a high accuracy, paving the way for AI to serve as an adjunct or independent tool alongside oral and maxillofacial pathologists for detecting and grading oral epithelial dysplasia.

Federated Learning in radiomics: A comprehensive meta-survey on medical image analysis

Federated Learning (FL) has emerged as a promising approach for collaborative medical image analysis while preserving data privacy, making it particularly suitable for radiomics tasks. This paper presents a systematic meta-analysis of recent surveys on Federated Learning in Medical Imaging (FL-MI), published in reputable venues over the past five years. We adopt the PRISMA methodology, categorizing and analyzing the existing body of research in FL-MI. Our analysis identifies common trends, challenges, and emerging strategies for implementing FL in medical imaging, including handling data heterogeneity, privacy concerns, and model performance in non-IID settings. The paper also highlights the most widely used datasets and a comparison of adopted machine learning models. Moreover, we examine FL frameworks in FL-MI applications, such as tumor detection, organ segmentation, and disease classification. We identify several research gaps, including the need for more robust privacy protection. Our findings provide a comprehensive overview of the current state of FL-MI and offer valuable directions for future research and development in this rapidly evolving field.

A comparison of an integrated and image-only deep learning model for predicting the disappearance of indeterminate pulmonary nodules

BACKGROUND

Indeterminate pulmonary nodules (IPNs) require follow-up CT to assess potential growth; however, benign nodules may disappear. Accurately predicting whether IPNs will resolve is a challenge for radiologists. Therefore, we aim to utilize deep-learning (DL) methods to predict the disappearance of IPNs.

MATERIAL AND METHODS

This retrospective study utilized data from the Dutch-Belgian Randomized Lung Cancer Screening Trial (NELSON) and Imaging in Lifelines (ImaLife) cohort. Participants underwent follow-up CT to determine the evolution of baseline IPNs. The NELSON data was used for model training. External validation was performed in ImaLife. We developed integrated DL-based models that incorporated CT images and demographic data (age, sex, smoking status, and pack years). We compared the performance of integrated methods with those limited to CT images only and calculated sensitivity, specificity, and area under the receiver operating characteristic curve (AUC). From a clinical perspective, ensuring high specificity is critical, as it minimizes false predictions of non-resolving nodules that should be monitored for evolution on follow-up CTs. Feature importance was calculated using SHapley Additive exPlanations (SHAP) values.

RESULTS

The training dataset included 840 IPNs (134 resolving) in 672 participants. The external validation dataset included 111 IPNs (46 resolving) in 65 participants. On the external validation set, the performance of the integrated model (sensitivity, 0.50; 95 % CI, 0.35-0.65; specificity, 0.91; 95 % CI, 0.80-0.96; AUC, 0.82; 95 % CI, 0.74-0.90) was comparable to that solely trained on CT image (sensitivity, 0.41; 95 % CI, 0.27-0.57; specificity, 0.89; 95 % CI, 0.78-0.95; AUC, 0.78; 95 % CI, 0.69-0.86; P = 0.39). The top 10 most important features were all image related.

CONCLUSION

Deep learning-based models can predict the disappearance of IPNs with high specificity. Integrated models using CT scans and clinical data had comparable performance to those using only CT images.

Enhancing multimodal medical image analysis with Slice-Fusion: A novel fusion approach to address modality imbalance

BACKGROUND AND OBJECTIVE

In recent times, medical imaging analysis (MIA) has seen an increasing interest due to its core application in computer-aided diagnosis systems (CADs). A modality in MIA refers to a specific technology used to produce human body images, such as MRI, CT scans, or X-rays. Each modality presents unique challenges and characteristics, often leading to imbalances within datasets. This significant challenge impedes model training and generalization due to the varying convergence rates of different modalities and the suppression of gradients in less dominant modalities.

METHODS

This paper proposes a novel fusion approach, and we named it Slice-Fusion. The proposed approach aims to mitigate the modality imbalance problem by implementing a "Modality-Specific-Balancing-Factor" fusion strategy. Furthermore, it incorporates an auxiliary (uni-modal) task that generates balanced modality pairs based on the image orientations of different modalities. Subsequently, a novel multimodal classification framework is presented to learn from the generated balanced modalities. The effectiveness of the proposed approach is evaluated through comparative assessments on a publicly available BraTS2021 dataset. The results demonstrate the efficiency of Slice-Fusion in resolving the modality imbalance problem. By enhancing the representation of balanced features and reducing modality bias, this approach holds promise for advancing visual health informatics and facilitating more accurate and reliable medical image analysis.

RESULTS

In the experiment section, three diverse experiments are conducted such as i) Fusion Loss Metrics Evaluation, ii) Classification, and iii) Visual Health Informatics. Notably, the proposed approach achieved an F1-Score of (100%, 81.25%) on the training and validation sets for the classification generalization task. In addition to the Slice-Fusion's out-performance, the study also created a new modality-aligned dataset (a highly balanced and informative modality-specific image collection) that aids further research and improves MIA's robustness. These advancements not only enhance the capability of medical diagnostic tools but also create opportunities for future innovations in the field.

CONCLUSION

This study contributes to advancing medical image analysis, such as effective modality fusion, image reconstruction, comparison, and glioma classification, facilitating more accurate and reliable results, and holds promise for further advancements in visual health informatics.

A device-dependent auto-segmentation method based on combined generalized and single-device datasets

BACKGROUND

Although generalized-dataset-based auto-segmentation models that consider various computed tomography (CT) scanners have shown great clinical potential, their application to medical images from unseen scanners remains challenging because of device-dependent image features.

PURPOSE

This study aims to investigate the performance of a device-dependent auto-segmentation model based on a combined dataset of a generalized dataset and single CT scanner dataset.

METHOD

We constructed two training datasets for 21 chest and abdominal organs. The generalized dataset comprised 1203 publicly available multi-scanner data. The device-dependent dataset comprised 1253 data, including the 1203 multi-CT scanner data and 50 single CT scanner data. Using these datasets, the generalized-dataset-based model (GDSM) and the device-dependent-dataset-based model (DDSM) were trained. The models were trained using nnU-Net and tested on ten data samples from a single CT scanner. The evaluation metrics included the Dice similarity coefficient (DSC), the Hausdorff distance (HD), and the average symmetric surface distance (ASSD), which were used to assess the overall performance of the models. In addition, DSCdiff, HDratio, and ASSDratio, which are variations of the three existing metrics, were used to compare the performance of the models across different organs.

RESULT

For the average DSC, the GDSM and DDSM had values of 0.9251 and 0.9323, respectively. For the average HD, the GDSM and DDSM had values of 10.66 and 9.139 mm, respectively; for the average ASSD, the GDSM and DDSM had values of 0.8318 and 0.6656 mm, respectively. Compared with the GDSM, the DDSM showed consistent performance improvements of 0.78%, 14%, and 20% for the DSC, HD, and ASSD metrics, respectively. In addition, compared with the GDSM, the DDSM had better DSCdiff values in 14 of 21 tested organs, better HDratio values in 13 of 21 tested organs, and better ASSDratio values in 14 of 21 tested organs. The three averages of the variant metrics were all better for the DDSM than for the GDSM.

CONCLUSION

The results suggest that combining the generalized dataset with a single scanner dataset resulted in an overall improvement in model performance for that device image.

Diagnostic performance of deep learning-assisted [18F]FDG PET imaging for Alzheimer's disease: a systematic review and meta-analysis

PURPOSE

This study aims to calculate the diagnostic performance of deep learning (DL)-assisted 18F-fluorodeoxyglucose ([18F]FDG) PET imaging in Alzheimer's disease (AD).

METHODS

The Ovid MEDLINE, Ovid Embase, Web of Science Core Collection, Cochrane, and IEEE Xplore databases were searched for related studies from inception to May 24, 2024. We included original studies that developed a DL algorithm for [18F]FDG PET imaging to assess diagnostic performance in classifying AD, mild cognitive impairment (MCI), and normal control (NC). A bivariate random-effects model was employed to assess the area under the curve (AUC).

RESULTS

We identified 36 studies that met the inclusion criteria. Of these, 35 studies distinguished AD from NC, with a pooled AUC of 0.98 (95% CI: 0.96-0.99). Thirteen studies distinguished AD from MCI, with a pooled AUC of 0.95 (95% CI: 0.92-0.96). Nineteen studies distinguished MCI from NC, with a pooled AUC of 0.94 (95% CI: 0.91-0.95). Additionally, we found large amounts of heterogeneity across studies which could be partially attributed to variations in DL methods and imaging modalities.

CONCLUSION

This systematic review and meta-analysis shows that DL-assisted [18F]FDG PET imaging has high diagnostic performance in identifying AD. The significant heterogeneity among studies underscores the necessity for future research to incorporate external validation, utilize large sample size, and adhere to rigorous guideline to provide robust support for clinical decision-making.

Exploring the application of deep learning methods for polygenic risk score estimation

Background. Polygenic risk scores (PRS) summarise genetic information into a single number with clinical and research uses. Deep learning (DL) has revolutionised multiple fields, however, the impact of DL on PRSs has been less significant. We explore how DL can improve the generation of PRSs.Methods. We train DL models on known PRSs using UK Biobank data. We explore whether the models can recreate human programmed PRSs, including using a single model to generate multiple PRSs, and DL difficulties in PRS generation. We investigate how DL can compensate for missing data and constraints on performance.Results. We demonstrate almost perfect generation of multiple PRSs with little loss of performance with reduced quantity of training data. For an example set of missing SNPs the DL model produces predictions that enable separation of cases from population samples with an area under the receiver operating characteristic curve of 0.847 (95% CI: 0.828-0.864) compared to 0.798 (95% CI: 0.779-0.818) for the PRS.Conclusions. DL can accurately generate PRSs, including with one model for multiple PRSs. The models are transferable and have high longevity. With certain missing SNPs the DL models can improve on PRS generation; further improvements would likely require additional input data.

Federated Active Learning Framework for Efficient Annotation Strategy in Skin-Lesion Classification

Federated learning (FL) enables multiple institutes to train models collaboratively without sharing private data. Current FL research focuses on communication efficiency, privacy protection, and personalization and assumes that the data of FL have already been ideally collected. However, in medical scenarios, data annotation demands both expertise and intensive labor, which is a critical problem in FL. Active learning (AL) has shown promising performance in reducing the number of data annotations in medical image analysis. We propose a federated AL framework in which AL is executed periodically and interactively under FL. We exploit a local model in each hospital and a global model acquired from FL to construct an ensemble. We use ensemble entropy-based AL as an efficient data-annotation strategy in FL. Therefore, our federated AL framework can decrease the amount of annotated data and preserve patient privacy while maintaining the performance of FL. To our knowledge, this federated AL framework applied to medical images has not been previously reported. We validated our framework on real-world dermoscopic datasets. Using only 50% of samples, our framework was able to achieve state-of-the-art performance on a skin-lesion classification task. Our framework performed better than several state-of-the-art AL methods under FL and achieved comparable performance with full-data FL.

Deep learning-based malaria parasite detection: convolutional neural networks model for accurate species identification of Plasmodium falciparum and Plasmodium vivax

Accurate malaria diagnosis with precise identification of Plasmodium species is crucial for an effective treatment. While microscopy is still the gold standard in malaria diagnosis, it relies heavily on trained personnel. Artificial intelligence (AI) advances, particularly convolutional neural networks (CNNs), have significantly improved diagnostic capabilities and accuracy by enabling the automated analysis of medical images. Previous models efficiently detected malaria parasites in red blood cells but had difficulty differentiating between species. We propose a CNN-based model for classifying cells infected by P. falciparum, P. vivax, and uninfected white blood cells from thick blood smears. Our best-performing model utilizes a seven-channel input and correctly predicted 12,876 out of 12,954 cases. We also generated a cross-validation confusion matrix that showed the results of five iterations, achieving 63,654 out of 64,126 true predictions. The model's accuracy reached 99.51%, a precision of 99.26%, a recall of 99.26%, a specificity of 99.63%, an F1 score of 99.26%, and a loss of 2.3%. We are now developing a system based on real-world quality images to create a comprehensive detection tool for remote regions where trained microscopists are unavailable.

Systematic Review of Hybrid Vision Transformer Architectures for Radiological Image Analysis

Vision transformer (ViT)and convolutional neural networks (CNNs) each possess distinct strengths in medical imaging: ViT excels in capturing long-range dependencies through self-attention, while CNNs are adept at extracting local features via spatial convolution filters. While ViT may struggle with capturing detailed local spatial information, critical for tasks like anomaly detection in medical imaging, shallow CNNs often fail to effectively abstract global context. This study aims to explore and evaluate hybrid architectures that integrate ViT and CNN to leverage their complementary strengths for enhanced performance in medical vision tasks, such as segmentation, classification, reconstruction, and prediction. Following PRISMA guideline, a systematic review was conducted on 34 articles published between 2020 and Sept. 2024. These articles proposed novel hybrid ViT-CNN architectures specifically for medical imaging tasks in radiology. The review focused on analyzing architectural variations, merging strategies between ViT and CNN, innovative applications of ViT, and efficiency metrics including parameters, inference time (GFlops), and performance benchmarks. The review identified that integrating ViT and CNN can mitigate the limitations of each architecture offering comprehensive solutions that combine global context understanding with precise local feature extraction. We benchmarked the articles based on architectural variations, merging strategies, innovative uses of ViT, and efficiency metrics (number of parameters, inference time (GFlops), and performance), and derived a ranked list. By synthesizing current literature, this review defines fundamental concepts of hybrid vision transformers and highlights emerging trends in the field. It provides a clear direction for future research aimed at optimizing the integration of ViT and CNN for effective utilization in medical imaging, contributing to advancements in diagnostic accuracy and image analysis. We performed systematic review of hybrid vision transformer architecture using PRISMA guideline and performed thorough comparative analysis to benchmark the architectures.

Evaluation of accuracy of deep learning and conventional neural network algorithms in detection of dental implant type using intraoral radiographic images: A systematic review and meta-analysis

STATEMENT OF PROBLEM

With the growing importance of implant brand detection in clinical practice, the accuracy of machine learning algorithms in implant brand detection has become a subject of research interest. Recent studies have shown promising results for the use of machine learning in implant brand detection. However, despite these promising findings, a comprehensive evaluation of the accuracy of machine learning in implant brand detection is needed.

PURPOSE

The purpose of this systematic review and meta-analysis was to assess the accuracy, sensitivity, and specificity of deep learning algorithms in implant brand detection using 2-dimensional images such as from periapical or panoramic radiographs.

MATERIAL AND METHODS

Electronic searches were conducted in PubMed, Embase, Scopus, Scopus Secondary, and Web of Science databases. Studies that met the inclusion criteria were assessed for quality using the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) tool. Meta-analyses were performed using the random-effects model to estimate the pooled performance measures and the 95% confidence intervals (CIs) using STATA v.17.

RESULTS

Thirteen studies were selected for the systematic review, and 3 were used in the meta-analysis. The meta-analysis of the studies found that the overall accuracy of CNN algorithms in detecting dental implants in radiographic images was 95.63%, with a sensitivity of 94.55% and a specificity of 97.91%. The highest reported accuracy was 99.08% for CNN Multitask ResNet152 algorithm, and sensitivity and specificity were 100.00% and 98.70% respectively for the deep CNN (Neuro-T version 2.0.1) algorithm with the Straumann SLActive BLT implant brand. All studies had a low risk of bias.

CONCLUSIONS

The highest accuracy and sensitivity were reported in studies using CNN Multitask ResNet152 and deep CNN (Neuro-T version 2.0.1) algorithms.

Researching public health datasets in the era of deep learning: a systematic literature review

Objective: Explore deep learning applications in predictive analytics for public health data, identify challenges and trends, and then understand the current landscape. Materials and Methods: A systematic literature review was conducted in June 2023 to search articles on public health data in the context of deep learning, published from the inception of medical and computer science databases through June 2023. The review focused on diverse datasets, abstracting applications, challenges, and advancements in deep learning. Results: 2004 articles were reviewed, identifying 14 disease categories. Observed trends include explainable-AI, patient embedding learning, and integrating different data sources and employing deep learning models in health informatics. Noted challenges were technical reproducibility and handling sensitive data. Discussion: There has been a notable surge in deep learning applications on public health data publications since 2015. Consistent deep learning applications and models continue to be applied across public health data. Despite the wide applications, a standard approach still does not exist for addressing the outstanding challenges and issues in this field. Conclusion: Guidelines are needed for applying deep learning and models in public health data to improve FAIRness, efficiency, transparency, comparability, and interoperability of research. Interdisciplinary collaboration among data scientists, public health experts, and policymakers is needed to harness the full potential of deep learning.

Classification and diagnosis of cervical lesions based on colposcopy images using deep fully convolutional networks: A man-machine comparison cohort study

Colposcopy is an important technique in the diagnosis of cervical cancer. The development of computer-aided diagnosis methods can mitigate the shortage of colposcopists and improve the accuracy and efficiency of colposcopy examinations in China. This study proposes the Dense-U-Net model for colposcopy image recognition. This was a man-machine comparison cohort study. It presents a novel artificial intelligence (AI) model for the diagnosis of cervical lesions through colposcopy images using a Dense-U-Net image semantic segmentation algorithm. The Dense-U-Net model was created by applying the methods of "deepening the network structure," "applying dropout" and "max pooling." Moreover, image-based and population-based diagnostic performances of the AI algorithm and physicians with different levels of specialist experience were compared. In total, 2,475 participants were recruited, and 13,084 colposcopy images were included in this study. The diagnostic accuracy of the Dense-U-Net model increased significantly with increasing colposcopy images per patient. As the number of images in the training set increased, the diagnostic accuracy of the Dense-U-Net model for cervical intraepithelial neoplasm 3 or worse (CIN3+) diagnosis increased (P = 0.035). The rate of diagnostic accuracy (0.89 vs 0.85, P < 0.001) of CIN3+ lesions using the Dense-U-Net model was higher than that of expert colposcopists, and the missed diagnosis (0.06 vs 0.07, P = 0.002) and false positive (0.05 vs 0.08, P < 0.001) were lower. Moreover, Dense-U-Net is more accurate in diagnosing the type III cervical transformation zone, which is difficult to diagnose by experts (P < 0.001). The Dense-U-Net model also showed higher diagnostic accuracy for CIN3+ in an independent test set (P < 0.001). To diagnose the same 870 test images, the Dense-U-Net system took 1.76 ± 0.09 min, while the expert, senior, and junior colposcopists took 716.3 ± 49.76, 892.1 ± 92.30, and 3034.7 ± 259.51 min, respectively. The study successfully built a reliable, quick, and effective Dense-U-Net model to assist with colposcopy examinations.

Reconstruction of Optical Coherence Tomography Images from Wavelength Space Using Deep Learning

Conventional Fourier domain Optical Coherence Tomography (FD-OCT) systems depend on resampling into a wavenumber (k) domain to extract the depth profile. This either necessitates additional hardware resources or amplifies the existing computational complexity. Moreover, the OCT images also suffer from speckle noise, due to systemic reliance on low-coherence interferometry. We propose a streamlined and computationally efficient approach based on Deep Learning (DL) which enables reconstructing speckle-reduced OCT images directly from the wavelength (λ) domain. For reconstruction, two encoder-decoder styled networks, namely Spatial Domain Convolution Neural Network (SD-CNN) and Fourier Domain CNN (FD-CNN), are used sequentially. The SD-CNN exploits the highly degraded images obtained by Fourier transforming the (λ) domain fringes to reconstruct the deteriorated morphological structures along with suppression of unwanted noise. The FD-CNN leverages this output to enhance the image quality further by optimization in the Fourier domain (FD). We quantitatively and visually demonstrate the efficacy of the method in obtaining high-quality OCT images. Furthermore, we illustrate the computational complexity reduction by harnessing the power of DL models. We believe that this work lays the framework for further innovations in the realm of OCT image reconstruction.

The Neural Frontier of Future Medical Imaging: A Review of Deep Learning for Brain Tumor Detection

Brain tumor detection is crucial in medical research due to high mortality rates and treatment challenges. Early and accurate diagnosis is vital for improving patient outcomes, however, traditional methods, such as manual Magnetic Resonance Imaging (MRI) analysis, are often time-consuming and error-prone. The rise of deep learning has led to advanced models for automated brain tumor feature extraction, segmentation, and classification. Despite these advancements, comprehensive reviews synthesizing recent findings remain scarce. By analyzing over 100 research papers over past half-decade (2019-2024), this review fills that gap, exploring the latest methods and paradigms, summarizing key concepts, challenges, datasets, and offering insights into future directions for brain tumor detection using deep learning. This review also incorporates an analysis of previous reviews and targets three main aspects: feature extraction, segmentation, and classification. The results revealed that research primarily focuses on Convolutional Neural Networks (CNNs) and their variants, with a strong emphasis on transfer learning using pre-trained models. Other methods, such as Generative Adversarial Networks (GANs) and Autoencoders, are used for feature extraction, while Recurrent Neural Networks (RNNs) are employed for time-sequence modeling. Some models integrate with Internet of Things (IoT) frameworks or federated learning for real-time diagnostics and privacy, often paired with optimization algorithms. However, the adoption of eXplainable AI (XAI) remains limited, despite its importance in building trust in medical diagnostics. Finally, this review outlines future opportunities, focusing on image quality, underexplored deep learning techniques, expanding datasets, and exploring deeper learning representations and model behavior such as recurrent expansion to advance medical imaging diagnostics.

DreamOn: a data augmentation strategy to narrow the robustness gap between expert radiologists and deep learning classifiers

Purpose

Successful performance of deep learning models for medical image analysis is highly dependent on the quality of the images being analysed. Factors like differences in imaging equipment and calibration, as well as patient-specific factors such as movements or biological variability (e.g., tissue density), lead to a large variability in the quality of obtained medical images. Consequently, robustness against the presence of noise is a crucial factor for the application of deep learning models in clinical contexts.

Materials and methods

We evaluate the effect of various data augmentation strategies on the robustness of a ResNet-18 trained to classify breast ultrasound images and benchmark the performance against trained human radiologists. Additionally, we introduce DreamOn, a novel, biologically inspired data augmentation strategy for medical image analysis. DreamOn is based on a conditional generative adversarial network (GAN) to generate REM-dream-inspired interpolations of training images.

Results

We find that while available data augmentation approaches substantially improve robustness compared to models trained without any data augmentation, radiologists outperform models on noisy images. Using DreamOn data augmentation, we obtain a substantial improvement in robustness in the high noise regime.

Conclusions

We show that REM-dream-inspired conditional GAN-based data augmentation is a promising approach to improving deep learning model robustness against noise perturbations in medical imaging. Additionally, we highlight a gap in robustness between deep learning models and human experts, emphasizing the imperative for ongoing developments in AI to match human diagnostic expertise.

Deep Learning Assisted Plasmonic Dark-Field Microscopy for Super-Resolution Label-Free Imaging

Dark-field microscopy (DFM) is a widely used imaging tool, due to its high-contrast capability in imaging label-free specimens. Traditional DFM requires optical alignment to block the oblique illumination, and the resolution is diffraction-limited to the wavelength scale. In this work, we present deep-learning assisted plasmonic dark-field microscopy (DAPD), which is a single-frame super-resolution method using plasmonic dark-field (PDF) microscopy and deep-learning assisted image reconstruction. Specifically, we fabricated a designed PDF substrate with surface plasmon polaritons (SPPs) illuminating specimens on the substrate. Dark field images formed by scattered light from the specimen are further processed by a pretrained convolutional neural network (CNN) using a simulation dataset based on the designed substrate and parameters of the detection optics. We demonstrated a resolution enhancement of 2.8 times on various label-free objects with a large potential for future improvement. We highlight our technique as a compact alternative to traditional DFM with a significantly enhanced spatial resolution.

Explainable rotation-invariant self-supervised representation learning

This paper describes a method that can perform robust detection and classification in out-of-distribution rotated images in the medical domain. In real-world medical imaging tools, noise due to the rotation of the body part is frequently observed. This noise reduces the accuracy of AI-based classification and prediction models. Hence, it is important to develop models which are rotation invariant. To that end, the proposed method - RISC (rotation invariant self-supervised vision framework) addresses this issue of rotational corruption. We present state-of-the-art rotation-invariant classification results and provide explainability for the performance in the domain. The evaluation of the proposed method is carried out on real-world adversarial examples in Medical Imagery-OrganAMNIST, RetinaMNIST and PneumoniaMNIST. It is observed that RISC outperforms the rotation-affected benchmark methods by obtaining 22\%, 17\% and 2\% accuracy boost on OrganAMNIST, PneumoniaMNIST and RetinaMNIST rotated baselines respectively. Further, explainability results are demonstrated. This methods paper describes:•a representation learning approach that can perform robust detection and classification in out-of-distribution rotated images in the medical domain.•It presents a method that incorporates self-supervised rotation invariance for correcting rotational corruptions.•GradCAM-based explainability for the rotational SSL pretext task and the downstream classification outcomes for the three benchmark datasets are presented.

Diagnostic accuracy of artificial intelligence for approximal caries on bitewing radiographs: A systematic review and meta-analysis

OBJECTIVES

This systematic review and meta-analysis aimed to investigate the diagnostic accuracy of Artificial Intelligence (AI) for approximal carious lesions on bitewing radiographs.

METHODS

This study included randomized controlled trials (RCTs) and non-randomized controlled trials (non-RCTs) reporting on the diagnostic accuracy of AI for approximal carious lesions on bitewing radiographs. The risk of bias was assessed using the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool. A systematic search was conducted on November 4, 2023, in PubMed, Cochrane, and Embase databases and an updated search was performed on August 28, 2024. The primary outcomes assessed were sensitivity, specificity, and overall accuracy. Sensitivity and specificity were pooled using a bivariate model.

RESULTS

Of the 2,442 studies identified, 21 met the inclusion criteria. The pooled sensitivity and specificity of AI were 0.94 (confidence interval (CI): ± 0.78-0.99) and 0.91 (CI: ± 0.84-0.95), respectively. The positive predictive value (PPV) ranged from 0.15 to 0.87, indicating a moderate capacity for identifying true positives among decayed teeth. The negative predictive value (NPV) ranged from 0.79 to 1.00, demonstrating a high ability to exclude healthy teeth. The diagnostic odds ratio was high, indicating strong overall diagnostic performance.

CONCLUSIONS

AI models demonstrate clinically acceptable diagnostic accuracy for approximal caries on bitewing radiographs. Although AI can be valuable for preliminary screening, positive findings should be verified by dental experts to prevent unnecessary treatments and ensure timely diagnosis. AI models are highly reliable in excluding healthy approximal surfaces.

CLINICAL SIGNIFICANCE

AI can assist dentists in detecting approximal caries on bitewing radiographs. However, expert supervision is required to prevent iatrogenic damage and ensure timely diagnosis.

DualStreamFoveaNet: A Dual Stream Fusion Architecture With Anatomical Awareness for Robust Fovea Localization

Accurate fovea localization is essential for analyzing retinal diseases to prevent irreversible vision loss. While current deep learning-based methods outperform traditional ones, they still face challenges such as the lack of local anatomical landmarks around the fovea, the inability to robustly handle diseased retinal images, and the variations in image conditions. In this paper, we propose a novel transformer-based architecture called DualStreamFoveaNet (DSFN) for multi-cue fusion. This architecture explicitly incorporates long-range connections and global features using retina and vessel distributions for robust fovea localization. We introduce a spatial attention mechanism in the dual-stream encoder to extract and fuse self-learned anatomical information, focusing more on features distributed along blood vessels and significantly reducing computational costs by decreasing token numbers. Our extensive experiments show that the proposed architecture achieves state-of-the-art performance on two public datasets and one large-scale private dataset. Furthermore, we demonstrate that the DSFN is more robust on both normal and diseased retina images and has better generalization capacity in cross-dataset experiments.

Deep-Learning Model for Quality Assessment of Urinary Bladder Ultrasound Images Using Multiscale and Higher-Order Processing

Autonomous ultrasound image quality assessment (US-IQA) is a promising tool to aid the interpretation by practicing sonographers and to enable the future robotization of ultrasound procedures. However, autonomous US-IQA has several challenges. Ultrasound images contain many spurious artifacts, such as noise due to handheld probe positioning, errors in the selection of probe parameters, and patient respiration during the procedure. Furthermore, these images are highly variable in appearance with respect to the individual patient's physiology. We propose to use a deep convolutional neural network (CNN), USQNet, which utilizes a multiscale and local-to-global second-order pooling (MS-L2GSoP) classifier to conduct the sonographer-like assessment of image quality. This classifier first extracts features at multiple scales to encode the interpatient anatomical variations, similar to a sonographer's understanding of anatomy. Then, it uses SoP in the intermediate layers (local) and at the end of the network (global) to exploit the second-order statistical dependency of MS structural and multiregion textural features. The L2GSoP will capture the higher-order relationships between different spatial locations and provide the seed for correlating local patches, much like a sonographer prioritizes regions across the image. We experimentally validated the USQNet for a new dataset of the human urinary bladder (UB) ultrasound images. The validation involved first with the subjective assessment by experienced radiologists' annotation, and then with state-of-the-art (SOTA) CNN networks for US-IQA and its ablated counterparts. The results demonstrate that USQNet achieves a remarkable accuracy of 92.4% and outperforms the SOTA models by 3%-14% while requiring comparable computation time.

The Rapidly Evolving Scenario of Acoustic Voice Analysis in Otolaryngology

The field of voice analysis has experienced significant transformations, evolving from basic perceptual assessments to the incorporation of advanced digital signal processing and computational tools. This progression has facilitated a deeper understanding of the complex dynamics of vocal function, particularly through the use of acoustic voice analysis within a multidimensional evaluation framework. Traditionally, voice analysis relied on parameters such as fundamental frequency, jitter, shimmer, and noise-to-harmonic ratio, which, despite their utility, have faced criticism for variability and lack of robustness. Recent developments have led to a shift toward more reliable metrics such as cepstral measures, which offer improved accuracy in voice quality assessments. Furthermore, the integration of multiparametric constructs underscores a comprehensive approach to evaluating vocal quality, blending sustained vowels, and continuous speech analyses. Current trends in clinical practice increasingly favor these advanced measures over traditional parameters due to their greater reliability and clinical utility. Additionally, the emergence of artificial intelligence (AI), particularly deep learning, holds promise for revolutionizing voice analysis by enhancing diagnostic precision and enabling efficient, non-invasive screening methods. This shift toward AI-driven approaches signifies a potential paradigm change in voice health, suggesting a future where AI not only aids in diagnosis but also the early detection and treatment of voice-related pathologies.

Reconstructing 3D histological structures using machine learning (artificial intelligence) algorithms

BACKGROUND

Histomorphometry is currently the gold standard for bone microarchitectural examinations. This relies on two-dimensional (2D) sections to deduce the spatial properties of structures. Micromorphometric parameters are calculated from these sections based on the assumption of a plate-like 3D microarchitecture, resulting in the loss of 3D structure due to the destructive nature of classical histological processing.

MATERIALS AND METHODS

To overcome the limitation of histomorphometry and reconstruct the 3D architecture of bone core biopsy samples from 2D histological sections, bone core biopsy samples were decalcified and embedded in paraffin. Subsequently, 5 µm thick serial sections were stained with hematoxylin and eosin and scanned using a 3DHISTECH PANNORAMIC 1000 Digital Slide Scanner (3DHISTECH, Budapest, Hungary). A modified U‑Net architecture was trained to categorize tissues on the sections. LoFTR feature matching combined with affine transformations was employed to create the histologic reconstruction. Micromorphometric parameters were calculated using Bruker's CTAn software (v. 1.18.8.0, Bruker, Kontich, Belgium) for both histological and microCT datasets.

RESULTS

Our method achieved an overall accuracy of 95.26% (95% confidence interval (CI): [94.15%, 96.37%]) with an F‑score of 0.9320 (95% CI: [0.9211, 0.9429]) averaged across all classes. Correlation coefficients between micromorphometric parameters measured on microCT imaging and histological reconstruction showed a strong linear relationship, with Spearman's ρ‑values of 0.777, 0.717, 0.705, 0.666, and 0.687 for bone volume/tissue volume (BV/TV), bone surface/TV, trabecular pattern factor, trabecular thickness, and trabecular separation, respectively. Bland-Altman and mountain plots indicated good agreement between the methods for BV/TV measurements.

CONCLUSION

This method enables examination of tissue microarchitecture in 3D with an even higher resolution than microcomputed tomography (microCT), without losing information on cellularity. However, the limitation of this procedure is its destructive nature, which precludes subsequent mechanical testing of the sample or any further secondary measurements. Furthermore, the number of histological sections that can be created from a single sample is limited.

Artificial intelligence-assisted focused cardiac ultrasound training: A survey among undergraduate medical students

Objectives

Focused cardiac ultrasound (FoCUS) is increasingly applied in many specialities, and adequate education and training of physicians is therefore mandatory. This study aimed to assess the impact of artificial intelligence (AI)-assisted interactive focused cardiac ultrasound (FoCUS) teaching session on undergraduate medical students' confidence level and knowledge in cardiac ultrasound.

Methods

The AI-assisted interactive FoCUS teaching session was held during the 9th National Undergraduate Cardiovascular Conference in London in March 2023 and all undergraduate medical students were invited to attend, and 79 students enrolled and attended the training. Two workshops were conducted each over 3-hour period. Each workshop consisted of a theoretical lecture followed by a supervised hands-on session by experts, first workshop trained 39 students and the second workshop trained 40 students. The students' pre- and post-session knowledge and confidence levels were assessed by Likert-type-scale questionnaires filled in by the students before and immediately after the workshop.

Results

A total of 61 pre-session and 52 post-session questionnaires were completed. Confidence level in ultrasound skills increased significantly for all six domains after the workshop, with the greatest improvement seen in obtaining basic cardiac views (p < 0.001 for all six domains). Students strongly agreed about the effectiveness of the teaching session and supported the integration of ultrasound training into their medical curriculum.

Conclusions

AI-assisted interactive FoCUS training can be an effective and powerful tool to increase ultrasound skills and confidence levels of undergraduate medical students. Integration of such ultrasound courses into the medical curriculum should therefore be considered.

Prospective clinical evaluation of deep learning for ultrasonographic screening of abdominal aortic aneurysms

Abdominal aortic aneurysm (AAA) often remains undetected until rupture due to limited access to diagnostic ultrasound. This trial evaluated a deep learning (DL) algorithm to guide AAA screening by novice nurses with no prior ultrasonography experience. Ten nurses performed 15 scans each on patients over 65, assisted by a DL object detection algorithm, and compared against physician-performed scans. Ultrasound scan quality, assessed by three blinded expert physicians, was the primary outcome. Among 184 patients, DL-guided novices achieved adequate scan quality in 87.5% of cases, comparable to the 91.3% by physicians (p = 0.310). The DL model predicted AAA with an AUC of 0.975, 100% sensitivity, and 97.8% specificity, with a mean absolute error of 2.8 mm in predicting aortic width compared to physicians. This study demonstrates that DL-guided POCUS has the potential to democratize AAA screening, offering performance comparable to experienced physicians and improving early detection.

Automated Neural Architecture Search for Cardiac Amyloidosis Classification from [18F]-Florbetaben PET Images

Medical image classification using convolutional neural networks (CNNs) is promising but often requires extensive manual tuning for optimal model definition. Neural architecture search (NAS) automates this process, reducing human intervention significantly. This study applies NAS to [18F]-Florbetaben PET cardiac images for classifying cardiac amyloidosis (CA) sub-types (amyloid light chain (AL) and transthyretin amyloid (ATTR)) and controls. Following data preprocessing and augmentation, an evolutionary cell-based NAS approach with a fixed network macro-structure is employed, automatically deriving cells' micro-structure. The algorithm is executed five times, evaluating 100 mutating architectures per run on an augmented dataset of 4048 images (originally 597), totaling 5000 architectures evaluated. The best network (NAS-Net) achieves 76.95% overall accuracy. K-fold analysis yields mean ± SD percentages of sensitivity, specificity, and accuracy on the test dataset: AL subjects (98.7 ± 2.9, 99.3 ± 1.1, 99.7 ± 0.7), ATTR-CA subjects (93.3 ± 7.8, 78.0 ± 2.9, 70.9 ± 3.7), and controls (35.8 ± 14.6, 77.1 ± 2.0, 96.7 ± 4.4). NAS-derived network performance rivals manually determined networks in the literature while using fewer parameters, validating its automatic approach's efficacy.

Application of artificial intelligence in lung cancer screening: A real-world study in a Chinese physical examination population

BACKGROUND

With the rapid increase of chest computed tomography (CT) images, the workload faced by radiologists has increased dramatically. It is undeniable that the use of artificial intelligence (AI) image-assisted diagnosis system in clinical treatment is a major trend in medical development. Therefore, in order to explore the value and diagnostic accuracy of the current AI system in clinical application, we aim to compare the detection and differentiation of benign and malignant pulmonary nodules between AI system and physicians, so as to provide a theoretical basis for clinical application.

METHODS

Our study encompassed a cohort of 23 336 patients who underwent chest low-dose spiral CT screening for lung cancer at the Health Management Center of West China Hospital. We conducted a comparative analysis between AI-assisted reading and manual interpretation, focusing on the detection and differentiation of benign and malignant pulmonary nodules.

RESULTS

The AI-assisted reading exhibited a significantly higher screening positive rate and probability of diagnosing malignant pulmonary nodules compared with manual interpretation (p < 0.001). Moreover, AI scanning demonstrated a markedly superior detection rate of malignant pulmonary nodules compared with manual scanning (97.2% vs. 86.4%, p < 0.001). Additionally, the lung cancer detection rate was substantially higher in the AI reading group compared with the manual reading group (98.9% vs. 90.3%, p < 0.001).

CONCLUSIONS

Our findings underscore the superior screening positive rate and lung cancer detection rate achieved through AI-assisted reading compared with manual interpretation. Thus, AI exhibits considerable potential as an adjunctive tool in lung cancer screening within clinical practice settings.

An Automated Vertebrae Localization, Segmentation, and Osteoporotic Compression Fracture Detection Pipeline for Computed Tomographic Imaging

Osteoporosis is the most common chronic metabolic bone disease worldwide. Vertebral compression fracture (VCF) is the most common type of osteoporotic fracture. Approximately 700,000 osteoporotic VCFs are diagnosed annually in the USA alone, resulting in an annual economic burden of ~$13.8B. With an aging population, the rate of osteoporotic VCFs and their associated burdens are expected to rise. Those burdens include pain, functional impairment, and increased medical expenditure. Therefore, it is of utmost importance to develop an analytical tool to aid in the identification of VCFs. Computed Tomography (CT) imaging is commonly used to detect occult injuries. Unlike the existing VCF detection approaches based on CT, the standard clinical criteria for determining VCF relies on the shape of vertebrae, such as loss of vertebral body height. We developed a novel automated vertebrae localization, segmentation, and osteoporotic VCF detection pipeline for CT scans using state-of-the-art deep learning models to bridge this gap. To do so, we employed a publicly available dataset of spine CT scans with 325 scans annotated for segmentation, 126 of which also graded for VCF (81 with VCFs and 45 without VCFs). Our approach attained 96% sensitivity and 81% specificity in detecting VCF at the vertebral-level, and 100% accuracy at the subject-level, outperforming deep learning counterparts tested for VCF detection without segmentation. Crucially, we showed that adding predicted vertebrae segments as inputs significantly improved VCF detection at both vertebral and subject levels by up to 14% Sensitivity and 20% Specificity (p-value = 0.028).

Deep learning based detection and classification of fetal lip in ultrasound images

OBJECTIVES

Fetal cleft lip is a common congenital defect. Considering the delicacy and difficulty of observing fetal lips, we have utilized deep learning technology to develop a new model aimed at quickly and accurately assessing the development of fetal lips during prenatal examinations. This model can detect ultrasound images of the fetal lips and classify them, aiming to provide a more objective prediction for the development of fetal lips.

METHODS

This study included 632 pregnant women in their mid-pregnancy stage, who underwent ultrasound examinations of the fetal lips, collecting both normal and abnormal fetal lip ultrasound images. To improve the accuracy of the detection and classification of fetal lips, we proposed and validated the Yolov5-ECA model.

RESULTS

The experimental results show that, compared with the currently popular 10 models, our model achieved the best results in the detection and classification of fetal lips. In terms of the detection of fetal lips, the mean average precision (mAP) at 0.5 and mAP at 0.5:0.95 were 0.920 and 0.630, respectively. In the classification of fetal lip ultrasound images, the accuracy reached 0.925.

CONCLUSIONS

The deep learning algorithm has accuracy consistent with manual evaluation in the detection and classification process of fetal lips. This automated recognition technology can provide a powerful tool for inexperienced young doctors, helping them to accurately conduct examinations and diagnoses of fetal lips.

Automatic classification of Giardia infection from stool microscopic images using deep neural networks

Introduction

Giardiasis is a common intestinal infection caused by the Giardia lamblia parasite, and its rapid and accurate diagnosis is crucial for effective treatment. The automatic classification of Giardia infection from stool microscopic images plays a vital role in this diagnosis process. In this study, we applied deep learning-based models to automatically classify stool microscopic images into three categories, namely, normal, cyst, and trophozoite.

Methods

Unlike previous studies focused on images acquired from drinking water samples, we specifically targeted stool samples. In this regard, we collected a dataset of 1610 microscopic digital images captured by a smartphone with a resolution of 2340 × 1080 pixels from stool samples under the Nikon YS100 microscope. First, we applied CLAHE (Contrast Limited Adaptive Histogram Equalization) histogram equalization a method to enhance the image quality and contrast. We employed three deep learning models, namely Xception, ResNet-50, and EfficientNet-B0, to evaluate their classification performance. Each model was trained on the dataset of microscopic images and fine-tuned using transfer learning techniques.

Results

Among the three deep learning models, EfficientNet-B0 demonstrated superior performance in classifying Giardia lamblia parasites from stool microscopic images. The model achieved precision, accuracy, recall, specificity, and F1-score values of 0.9599, 0.9629, 0.9619, 0.9821, and 0.9607, respectively.

Conclusion

The EfficientNet-B0 showed promising results in accurately identifying normal, cyst, and trophozoite forms of Giardia lamblia parasites. This automated classification approach can provide valuable assistance to laboratory science experts and parasitologists in the rapid and accurate diagnosis of giardiasis, ultimately improving patient care and treatment outcomes.

Data-driven classification and explainable-AI in the field of lung imaging

Detecting lung diseases in medical images can be quite challenging for radiologists. In some cases, even experienced experts may struggle with accurately diagnosing chest diseases, leading to potential inaccuracies due to complex or unseen biomarkers. This review paper delves into various datasets and machine learning techniques employed in recent research for lung disease classification, focusing on pneumonia analysis using chest X-ray images. We explore conventional machine learning methods, pretrained deep learning models, customized convolutional neural networks (CNNs), and ensemble methods. A comprehensive comparison of different classification approaches is presented, encompassing data acquisition, preprocessing, feature extraction, and classification using machine vision, machine and deep learning, and explainable-AI (XAI). Our analysis highlights the superior performance of transfer learning-based methods using CNNs and ensemble models/features for lung disease classification. In addition, our comprehensive review offers insights for researchers in other medical domains too who utilize radiological images. By providing a thorough overview of various techniques, our work enables the establishment of effective strategies and identification of suitable methods for a wide range of challenges. Currently, beyond traditional evaluation metrics, researchers emphasize the importance of XAI techniques in machine and deep learning models and their applications in classification tasks. This incorporation helps in gaining a deeper understanding of their decision-making processes, leading to improved trust, transparency, and overall clinical decision-making. Our comprehensive review serves as a valuable resource for researchers and practitioners seeking not only to advance the field of lung disease detection using machine learning and XAI but also from other diverse domains.

Synthetic CT for gamma knife radiosurgery dose calculation: A feasibility study

PURPOSE

To determine if MRI-based synthetic CTs (sCT), generated with no predefined pulse sequence, can be used for inhomogeneity correction in routine gamma knife radiosurgery (GKRS) treatment planning dose calculation.

METHODS

Two sets of sCTs were generated from T1post and T2 images using cycleGAN. Twenty-eight patients (18 training, 10 validation) were retrospectively selected. The image quality of the generated sCTs was compared with the original CT (oCT) regarding the HU value preservation using histogram comparison, RMSE and MAE, and structural integrity. Dosimetric comparisons were also made among GKRS plans from 3 calculation approaches: TMR10 (oCT), and convolution (oCT and sCT), at four locations: original disease site, bone/tissue interface, air/tissue interface, and mid-brain.

RESULTS

The study showed that sCTs and oCTs' HU were similar, with T2-sCT performing better. TMR10 significantly underdosed the target by a mean of 5.4% compared to the convolution algorithm. There was no significant difference in convolution algorithm shot time between the oCT and sCT generated with T2. The highest and lowest dosimetric differences between the two CTs were observed in the bone and air interface, respectively. Dosimetric differences of 3.3% were observed in sCT predicted from MRI with stereotactic frames, which was not included in the training sets.

CONCLUSIONS

MRI-based sCT can be utilized for GKRS convolution dose calculation without the unnecessary radiation dose, and sCT without metal artifacts could be generated in framed cases. Larger datasets inclusive of all pulse sequences can improve the training set. Further investigation and validation studies are needed before clinical implementation.

CHNet: A multi-task global-local Collaborative Hybrid Network for KRAS mutation status prediction in colorectal cancer

Accurate prediction of Kirsten rat sarcoma (KRAS) mutation status is crucial for personalized treatment of advanced colorectal cancer patients. However, despite the excellent performance of deep learning models in certain aspects, they often overlook the synergistic promotion among multiple tasks and the consideration of both global and local information, which can significantly reduce prediction accuracy. To address these issues, this paper proposes an innovative method called the Multi-task Global-Local Collaborative Hybrid Network (CHNet) aimed at more accurately predicting patients' KRAS mutation status. CHNet consists of two branches that can extract global and local features from segmentation and classification tasks, respectively, and exchange complementary information to collaborate in executing these tasks. Within the two branches, we have designed a Channel-wise Hybrid Transformer (CHT) and a Spatial-wise Hybrid Transformer (SHT). These transformers integrate the advantages of both Transformer and CNN, employing cascaded hybrid attention and convolution to capture global and local information from the two tasks. Additionally, we have created an Adaptive Collaborative Attention (ACA) module to facilitate the collaborative fusion of segmentation and classification features through guidance. Furthermore, we introduce a novel Class Activation Map (CAM) loss to encourage CHNet to learn complementary information between the two tasks. We evaluate CHNet on the T2-weighted MRI dataset, and achieve an accuracy of 88.93% in KRAS mutation status prediction, which outperforms the performance of representative KRAS mutation status prediction methods. The results suggest that our CHNet can more accurately predict KRAS mutation status in patients via a multi-task collaborative facilitation and considering global-local information way, which can assist doctors in formulating more personalized treatment strategies for patients.

Single-Image-Based Deep Learning for Precise Atomic Defect Identification

Defect engineering is widely used to impart the desired functionalities on materials. Despite the widespread application of atomic-resolution scanning transmission electron microscopy (STEM), traditional methods for defect analysis are highly sensitive to random noise and human bias. While deep learning (DL) presents a viable alternative, it requires extensive amounts of training data with labeled ground truth. Herein, employing cycle generative adversarial networks (CycleGAN) and U-Nets, we propose a method based on a single experimental STEM image to tackle high annotation costs and image noise for defect detection. Not only atomic defects but also oxygen dopants in monolayer MoS2 are visualized. The method can be readily extended to other two-dimensional systems, as the training is based on unit-cell-level images. Therefore, our results outline novel ways to train the model with minimal data sets, offering great opportunities to fully exploit the power of DL in the materials science community.

A federated learning architecture for secure and private neuroimaging analysis

The amount of biomedical data continues to grow rapidly. However, collecting data from multiple sites for joint analysis remains challenging due to security, privacy, and regulatory concerns. To overcome this challenge, we use federated learning, which enables distributed training of neural network models over multiple data sources without sharing data. Each site trains the neural network over its private data for some time and then shares the neural network parameters (i.e., weights and/or gradients) with a federation controller, which in turn aggregates the local models and sends the resulting community model back to each site, and the process repeats. Our federated learning architecture, MetisFL, provides strong security and privacy. First, sample data never leave a site. Second, neural network parameters are encrypted before transmission and the global neural model is computed under fully homomorphic encryption. Finally, we use information-theoretic methods to limit information leakage from the neural model to prevent a "curious" site from performing model inversion or membership attacks. We present a thorough evaluation of the performance of secure, private federated learning in neuroimaging tasks, including for predicting Alzheimer's disease and for brain age gap estimation (BrainAGE) from magnetic resonance imaging (MRI) studies in challenging, heterogeneous federated environments where sites have different amounts of data and statistical distributions.

Linked Color Imaging with Artificial Intelligence Improves the Detection of Early Gastric Cancer

INTRODUCTION

Esophagogastroduodenoscopy is the most important tool to detect gastric cancer (GC). In this study, we developed a computer-aided detection (CADe) system to detect GC with white light imaging (WLI) and linked color imaging (LCI) modes and aimed to compare the performance of CADe with that of endoscopists.

METHODS

The system was developed based on the deep learning framework from 9,021 images in 385 patients between 2017 and 2020. A total of 116 LCI and WLI videos from 110 patients between 2017 and 2023 were used to evaluate per-case sensitivity and per-frame specificity.

RESULTS

The per-case sensitivity and per-frame specificity of CADe with a confidence level of 0.5 in detecting GC were 78.6% and 93.4% for WLI and 94.0% and 93.3% for LCI, respectively (p < 0.001). The per-case sensitivities of nonexpert endoscopists for WLI and LCI were 45.8% and 80.4%, whereas those of expert endoscopists were 66.7% and 90.6%, respectively. Regarding detectability between CADe and endoscopists, the per-case sensitivities for WLI and LCI were 78.6% and 94.0% in CADe, respectively, which were significantly higher than those for LCI in experts (90.6%, p = 0.004) and those for WLI and LCI in nonexperts (45.8% and 80.4%, respectively, p < 0.001); however, no significant difference for WLI was observed between CADe and experts (p = 0.134).

CONCLUSIONS

Our CADe system showed significantly better sensitivity in detecting GC when used in LCI compared with WLI mode. Moreover, the sensitivity of CADe using LCI is significantly higher than those of expert endoscopists using LCI to detect GC.

The Performance and Clinical Applicability of HER2 Digital Image Analysis in Breast Cancer: A Systematic Review

This systematic review aims to address the research gap in the performance of computational algorithms for the digital image analysis of HER2 images in clinical settings. While numerous studies have explored various aspects of these algorithms, there is a lack of comprehensive evaluation regarding their effectiveness in real-world clinical applications. We conducted a search of the Web of Science and PubMed databases for studies published from 31 December 2013 to 30 June 2024, focusing on performance effectiveness and components such as dataset size, diversity and source, ground truth, annotation, and validation methods. The study was registered with PROSPERO (CRD42024525404). Key questions guiding this review include the following: How effective are current computational algorithms at detecting HER2 status in digital images? What are the common validation methods and dataset characteristics used in these studies? Is there standardization of algorithm evaluations of clinical applications that can improve the clinical utility and reliability of computational tools for HER2 detection in digital image analysis? We identified 6833 publications, with 25 meeting the inclusion criteria. The accuracy rate with clinical datasets varied from 84.19% to 97.9%. The highest accuracy was achieved on the publicly available Warwick dataset at 98.8% in synthesized datasets. Only 12% of studies used separate datasets for external validation; 64% of studies used a combination of accuracy, precision, recall, and F1 as a set of performance measures. Despite the high accuracy rates reported in these studies, there is a notable absence of direct evidence supporting their clinical application. To facilitate the integration of these technologies into clinical practice, there is an urgent need to address real-world challenges and overreliance on internal validation. Standardizing study designs on real clinical datasets can enhance the reliability and clinical applicability of computational algorithms in improving the detection of HER2 cancer.

Comparative study of DCNN and image processing based classification of chest X-rays for identification of COVID-19 patients using fine-tuning

The conventional detection of COVID-19 by evaluating the CT scan images is tiresome, often experiences high inter-observer variability and uncertainty issues. This work proposes the automatic detection and classification of COVID-19 by analysing the chest X-ray images (CXR) with the deep convolutional neural network (DCNN) models through a fine-tuning and pre-training approach. CXR images pertaining to four health scenarios, namely, healthy, COVID-19, bacterial pneumonia and viral pneumonia, are considered and subjected to data augmentation. Two types of input datasets are prepared; in which dataset I contains the original image dataset categorised under four classes, whereas the original CXR images are subjected to image pre-processing via Contrast Limited Adaptive Histogram Equalisation (CLAHE) algorithm and Blackhat Morphological Operation (BMO) for devising the input dataset II. Both datasets are supplied as input to various DCNN models such as DenseNet, MobileNet, ResNet, VGG16, and Xception for achieving multi-class classification. It is observed that the classification accuracies are improved, and the classification errors are reduced with the image pre-processing. Overall, the VGG16 model resulted in better classification accuracies and reduced classification errors while accomplishing multi-class classification. Thus, the proposed work would assist the clinical diagnosis, and reduce the workload of the front-line healthcare workforce and medical professionals.

Exploring Publicly Accessible Optical Coherence Tomography Datasets: A Comprehensive Overview

Artificial intelligence has transformed medical diagnostic capabilities, particularly through medical image analysis. AI algorithms perform well in detecting abnormalities with a strong performance, enabling computer-aided diagnosis by analyzing the extensive amounts of patient data. The data serve as a foundation upon which algorithms learn and make predictions. Thus, the importance of data cannot be underestimated, and clinically corresponding datasets are required. Many researchers face a lack of medical data due to limited access, privacy concerns, or the absence of available annotations. One of the most widely used diagnostic tools in ophthalmology is Optical Coherence Tomography (OCT). Addressing the data availability issue is crucial for enhancing AI applications in the field of OCT diagnostics. This review aims to provide a comprehensive analysis of all publicly accessible retinal OCT datasets. Our main objective is to compile a list of OCT datasets and their properties, which can serve as an accessible reference, facilitating data curation for medical image analysis tasks. For this review, we searched through the Zenodo repository, Mendeley Data repository, MEDLINE database, and Google Dataset search engine. We systematically evaluated all the identified datasets and found 23 open-access datasets containing OCT images, which significantly vary in terms of size, scope, and ground-truth labels. Our findings indicate the need for improvement in data-sharing practices and standardized documentation. Enhancing the availability and quality of OCT datasets will support the development of AI algorithms and ultimately improve diagnostic capabilities in ophthalmology. By providing a comprehensive list of accessible OCT datasets, this review aims to facilitate better utilization and development of AI in medical image analysis.

Breast Cancer Detection and Analytics Using Hybrid CNN and Extreme Learning Machine

Early detection of breast cancer is essential for increasing survival rates, as it is one of the primary causes of death for women globally. Mammograms are extensively used by physicians for diagnosis, but selecting appropriate algorithms for image enhancement, segmentation, feature extraction, and classification remains a significant research challenge. This paper presents a computer-aided diagnosis (CAD)-based hybrid model combining convolutional neural networks (CNN) with a pruned ensembled extreme learning machine (HCPELM) to enhance breast cancer detection, segmentation, feature extraction, and classification. The model employs the rectified linear unit (ReLU) activation function to enhance data analytics after removing artifacts and pectoral muscles, and the HCPELM hybridized with the CNN model improves feature extraction. The hybrid elements are convolutional and fully connected layers. Convolutional layers extract spatial features like edges, textures, and more complex features in deeper layers. The fully connected layers take these features and combine them in a non-linear manner to perform the final classification. ELM performs classification and recognition tasks, aiming for state-of-the-art performance. This hybrid classifier is used for transfer learning by freezing certain layers and modifying the architecture to reduce parameters, easing cancer detection. The HCPELM classifier was trained using the MIAS database and evaluated against benchmark methods. It achieved a breast image recognition accuracy of 86%, outperforming benchmark deep learning models. HCPELM is demonstrating superior performance in early detection and diagnosis, thus aiding healthcare practitioners in breast cancer diagnosis.

Deep learning empowered breast cancer diagnosis: Advancements in detection and classification

Recent advancements in AI, driven by big data technologies, have reshaped various industries, with a strong focus on data-driven approaches. This has resulted in remarkable progress in fields like computer vision, e-commerce, cybersecurity, and healthcare, primarily fueled by the integration of machine learning and deep learning models. Notably, the intersection of oncology and computer science has given rise to Computer-Aided Diagnosis (CAD) systems, offering vital tools to aid medical professionals in tumor detection, classification, recurrence tracking, and prognosis prediction. Breast cancer, a significant global health concern, is particularly prevalent in Asia due to diverse factors like lifestyle, genetics, environmental exposures, and healthcare accessibility. Early detection through mammography screening is critical, but the accuracy of mammograms can vary due to factors like breast composition and tumor characteristics, leading to potential misdiagnoses. To address this, an innovative CAD system leveraging deep learning and computer vision techniques was introduced. This system enhances breast cancer diagnosis by independently identifying and categorizing breast lesions, segmenting mass lesions, and classifying them based on pathology. Thorough validation using the Curated Breast Imaging Subset of Digital Database for Screening Mammography (CBIS-DDSM) demonstrated the CAD system's exceptional performance, with a 99% success rate in detecting and classifying breast masses. While the accuracy of detection is 98.5%, when segmenting breast masses into separate groups for examination, the method's performance was approximately 95.39%. Upon completing all the analysis, the system's classification phase yielded an overall accuracy of 99.16% for classification. The potential for this integrated framework to outperform current deep learning techniques is proposed, despite potential challenges related to the high number of trainable parameters. Ultimately, this recommended framework offers valuable support to researchers and physicians in breast cancer diagnosis by harnessing cutting-edge AI and image processing technologies, extending recent advances in deep learning to the medical domain.

Classification of brain tumor types through MRIs using parallel CNNs and firefly optimization

Image segmentation is a critical and challenging endeavor in the field of medicine. A magnetic resonance imaging (MRI) scan is a helpful method for locating any abnormal brain tissue these days. It is a difficult undertaking for radiologists to diagnose and classify the tumor from several pictures. This work develops an intelligent method for accurately identifying brain tumors. This research investigates the identification of brain tumor types from MRI data using convolutional neural networks and optimization strategies. Two novel approaches are presented: the first is a novel segmentation technique based on firefly optimization (FFO) that assesses segmentation quality based on many parameters, and the other is a combination of two types of convolutional neural networks to categorize tumor traits and identify the kind of tumor. These upgrades are intended to raise the general efficacy of the MRI scan technique and increase identification accuracy. Using MRI scans from BBRATS2018, the testing is carried out, and the suggested approach has shown improved performance with an average accuracy of 98.6%.

Application and progress of artificial intelligence in radiation therapy dose prediction

Radiation therapy (RT) nowadays is a main treatment modality of cancer. To ensure the therapeutic efficacy of patients, accurate dose distribution is often required, which is a time-consuming and labor-intensive process. In addition, due to the differences in knowledge and experience among participants and diverse institutions, the predicted dose are often inconsistent. In last several decades, artificial intelligence (AI) has been applied in various aspects of RT, several products have been implemented in clinical practice and confirmed superiority. In this paper, we will review the research of AI in dose prediction, focusing on the progress in deep learning (DL).

Visual Knowledge Domain of Artificial Intelligence in Computed Tomography: A Review Based on Bibliometric Analysis

ABSTRACT

Artificial intelligence (AI)-assisted medical imaging technology is a new research area of great interest that has developed rapidly over the last decade. However, there has been no bibliometric analysis of published studies in this field. The present review focuses on AI-related studies on computed tomography imaging in the Web of Science database and uses CiteSpace and VOSviewer to generate a knowledge map and conduct the basic information analysis, co-word analysis, and co-citation analysis. A total of 7265 documents were included and the number of documents published had an overall upward trend. Scholars from the United States and China have made outstanding achievements, and there is a general lack of extensive cooperation in this field. In recent years, the research areas of great interest and difficulty have been the optimization and upgrading of algorithms, and the application of theoretical models to practical clinical applications. This review will help researchers understand the developments, research areas of great interest, and research frontiers in this field and provide reference and guidance for future studies.

Developing deep learning methods for classification of teeth in dental panoramic radiography

OBJECTIVES

We aimed to develop an artificial intelligence-based clinical dental decision-support system using deep-learning methods to reduce diagnostic interpretation error and time and increase the effectiveness of dental treatment and classification.

STUDY DESIGN

We compared the performance of 2 deep-learning methods, You Only Look Once V4 (YOLO-V4) and Faster Regions with the Convolutional Neural Networks (R-CNN), for tooth classification in dental panoramic radiography for tooth classification in dental panoramic radiography to determine which is more successful in terms of accuracy, time, and detection ability. Using a method based on deep-learning models trained on a semantic segmentation task, we analyzed 1200 panoramic radiographs selected retrospectively. In the classification process, our model identified 36 classes, including 32 teeth and 4 impacted teeth.

RESULTS

The YOLO-V4 method achieved a mean 99.90% precision, 99.18% recall, and 99.54% F1 score. The Faster R-CNN method achieved a mean 93.67% precision, 90.79% recall, and 92.21% F1 score. Experimental evaluations showed that the YOLO-V4 method outperformed the Faster R-CNN method in terms of accuracy of predicted teeth in the tooth classification process, speed of tooth classification, and ability to detect impacted and erupted third molars.

CONCLUSIONS

The YOLO-V4 method outperforms the Faster R-CNN method in terms of accuracy of tooth prediction, speed of detection, and ability to detect impacted third molars and erupted third molars. The proposed deep learning based methods can assist dentists in clinical decision making, save time, and reduce the negative effects of stress and fatigue in daily practice.

Tooth numbering and classification on bitewing radiographs: an artificial intelligence pilot study

OBJECTIVE

The aim of this study is to assess the efficacy of employing a deep learning methodology for the automated identification and enumeration of permanent teeth in bitewing radiographs. The experimental procedures and techniques employed in this study are described in the following section.

STUDY DESIGN

A total of 1248 bitewing radiography images were annotated using the CranioCatch labeling program, developed in Eskişehir, Turkey. The dataset has been partitioned into 3 subsets: training (n = 1000, 80% of the total), validation (n = 124, 10% of the total), and test (n = 124, 10% of the total) sets. The images were subjected to a 3 × 3 clash operation in order to enhance the clarity of the labeled regions.

RESULTS

The F1, sensitivity and precision results of the artificial intelligence model obtained using the Yolov5 architecture in the test dataset were found to be 0.9913, 0.9954, and 0.9873, respectively.

CONCLUSION

The utilization of numerical identification for teeth within deep learning-based artificial intelligence algorithms applied to bitewing radiographs has demonstrated notable efficacy. The utilization of clinical decision support system software, which is augmented by artificial intelligence, has the potential to enhance the efficiency and effectiveness of dental practitioners.

A Comprehensive Evaluation of Deep Learning Models on Knee MRIs for the Diagnosis and Classification of Meniscal Tears: A Systematic Review and Meta-Analysis

OBJECTIVES

This study delves into the cutting-edge field of deep learning techniques, particularly deep convolutional neural networks (DCNNs), which have demonstrated unprecedented potential in assisting radiologists and orthopedic surgeons in precisely identifying meniscal tears. This research aims to evaluate the effectiveness of deep learning models in recognizing, localizing, describing, and categorizing meniscal tears in magnetic resonance images (MRIs).

MATERIALS AND METHODS

This systematic review was rigorously conducted, strictly following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Extensive searches were conducted on MEDLINE (PubMed), Web of Science, Cochrane Library, and Google Scholar. All identified articles underwent a comprehensive risk of bias analysis. Predictive performance values were either extracted or calculated for quantitative analysis, including sensitivity and specificity. The meta-analysis was performed for all prediction models that identified the presence and location of meniscus tears.

RESULTS

This study's findings underscore that a range of deep learning models exhibit robust performance in detecting and classifying meniscal tears, in one case surpassing the expertise of musculoskeletal radiologists. Most studies in this review concentrated on identifying tears in the medial or lateral meniscus and even precisely locating tears-whether in the anterior or posterior horn-with exceptional accuracy, as demonstrated by AUC values ranging from 0.83 to 0.94.

CONCLUSIONS

Based on these findings, deep learning models have showcased significant potential in analyzing knee MR images by learning intricate details within images. They offer precise outcomes across diverse tasks, including segmenting specific anatomical structures and identifying pathological regions. Contributions: This study focused exclusively on DL models for identifying and localizing meniscus tears. It presents a meta-analysis that includes eight studies for detecting the presence of a torn meniscus and a meta-analysis of three studies with low heterogeneity that localize and classify the menisci. Another novelty is the analysis of arthroscopic surgery as ground truth. The quality of the studies was assessed against the CLAIM checklist, and the risk of bias was determined using the QUADAS-2 tool.

Deep learning for automatic detection of cephalometric landmarks on lateral cephalometric radiographs using the Mask Region-based Convolutional Neural Network: a pilot study

OBJECTIVE

We examined the effectiveness and feasibility of the Mask Region-based Convolutional Neural Network (Mask R-CNN) for automatic detection of cephalometric landmarks on lateral cephalometric radiographs (LCRs).

STUDY DESIGN

In total, 400 LCRs, each with 19 manually identified landmarks, were collected. Of this total, 320 images were randomly selected as the training dataset for Mask R-CNN, and the remaining 80 images were used for testing the automatic detection of the 19 cephalometric landmarks, for a total of 1520 landmarks. Detection rate, average error, and detection accuracy rate were calculated to assess Mask R-CNN performance.

RESULTS

Of the 1520 landmarks, 1494 were detected, for a detection rate of 98.29%. The average error, or linear deviation distance between the detected points and the originally marked points of each detected landmark, ranged from 0.56 to 9.51 mm, with an average of 2.19 mm. For detection accuracy rate, 649 landmarks (43.44%) had a linear deviation distance less than 1 mm, 1020 (68.27%) less than 2 mm, and 1281 (85.74%) less than 4 mm in deviation from the manually marked point. The average detection time was 1.48 seconds per image.

CONCLUSIONS

Deep learning Mask R-CNN shows promise in enhancing cephalometric analysis by automating landmark detection on LCRs, addressing the limitations of manual analysis, and demonstrating effectiveness and feasibility.