A scoping review of transfer learning research on medical image analysis using ImageNet

Application of artificial intelligence in cancer diagnosis and tumor nanomedicine

Cancer is a major health concern due to its high incidence and mortality rates. Advances in cancer research, particularly in artificial intelligence (AI) and deep learning, have shown significant progress. The swift evolution of AI in healthcare, especially in tools like computer-aided diagnosis, has the potential to revolutionize early cancer detection. This technology offers improved speed, accuracy, and sensitivity, bringing a transformative impact on cancer diagnosis, treatment, and management. This paper provides a concise overview of the application of artificial intelligence in the realms of medicine and nanomedicine, with a specific emphasis on the significance and challenges associated with cancer diagnosis. It explores the pivotal role of AI in cancer diagnosis, leveraging structured, unstructured, and multimodal fusion data. Additionally, the article delves into the applications of AI in nanomedicine sensors and nano-oncology drugs. The fundamentals of deep learning and convolutional neural networks are clarified, underscoring their relevance to AI-driven cancer diagnosis. A comparative analysis is presented, highlighting the accuracy and efficiency of traditional methods juxtaposed with AI-based approaches. The discussion not only assesses the current state of AI in cancer diagnosis but also delves into the challenges faced by AI in this context. Furthermore, the article envisions the future development direction and potential application of artificial intelligence in cancer diagnosis, offering a hopeful prospect for enhanced cancer detection and improved patient prognosis.

Novel Hybrid Quantum Architecture-Based Lung Cancer Detection Using Chest Radiograph and Computerized Tomography Images

Lung cancer, the second most common type of cancer worldwide, presents significant health challenges. Detecting this disease early is essential for improving patient outcomes and simplifying treatment. In this study, we propose a hybrid framework that combines deep learning (DL) with quantum computing to enhance the accuracy of lung cancer detection using chest radiographs (CXR) and computerized tomography (CT) images. Our system utilizes pre-trained models for feature extraction and quantum circuits for classification, achieving state-of-the-art performance in various metrics. Not only does our system achieve an overall accuracy of 92.12%, it also excels in other crucial performance measures, such as sensitivity (94%), specificity (90%), F1-score (93%), and precision (92%). These results demonstrate that our hybrid approach can more accurately identify lung cancer signatures compared to traditional methods. Moreover, the incorporation of quantum computing enhances processing speed and scalability, making our system a promising tool for early lung cancer screening and diagnosis. By leveraging the strengths of quantum computing, our approach surpasses traditional methods in terms of speed, accuracy, and efficiency. This study highlights the potential of hybrid computational technologies to transform early cancer detection, paving the way for wider clinical applications and improved patient care outcomes.

Advancing maxillofacial prosthodontics by using pre-trained convolutional neural networks: Image-based classification of the maxilla

PURPOSE

The study aimed to compare the performance of four pre-trained convolutional neural networks in recognizing seven distinct prosthodontic scenarios involving the maxilla, as a preliminary step in developing an artificial intelligence (AI)-powered prosthesis design system.

MATERIALS AND METHODS

Seven distinct classes, including cleft palate, dentulous maxillectomy, edentulous maxillectomy, reconstructed maxillectomy, completely dentulous, partially edentulous, and completely edentulous, were considered for recognition. Utilizing transfer learning and fine-tuned hyperparameters, four AI models (VGG16, Inception-ResNet-V2, DenseNet-201, and Xception) were employed. The dataset, consisting of 3541 preprocessed intraoral occlusal images, was divided into training, validation, and test sets. Model performance metrics encompassed accuracy, precision, recall, F1 score, area under the receiver operating characteristic curve (AUC), and confusion matrix.

RESULTS

VGG16, Inception-ResNet-V2, DenseNet-201, and Xception demonstrated comparable performance, with maximum test accuracies of 0.92, 0.90, 0.94, and 0.95, respectively. Xception and DenseNet-201 slightly outperformed the other models, particularly compared with InceptionResNet-V2. Precision, recall, and F1 scores exceeded 90% for most classes in Xception and DenseNet-201 and the average AUC values for all models ranged between 0.98 and 1.00.

CONCLUSIONS

While DenseNet-201 and Xception demonstrated superior performance, all models consistently achieved diagnostic accuracy exceeding 90%, highlighting their potential in dental image analysis. This AI application could help work assignments based on difficulty levels and enable the development of an automated diagnosis system at patient admission. It also facilitates prosthesis designing by integrating necessary prosthesis morphology, oral function, and treatment difficulty. Furthermore, it tackles dataset size challenges in model optimization, providing valuable insights for future research.

RadImageNet and ImageNet as Datasets for Transfer Learning in the Assessment of Dental Radiographs: A Comparative Study

Transfer learning (TL) is an alternative approach to the full training of deep learning (DL) models from scratch and can transfer knowledge gained from large-scale data to solve different problems. ImageNet, which is a publicly available large-scale dataset, is a commonly used dataset for TL-based image analysis; many studies have applied pre-trained models from ImageNet to clinical prediction tasks and have reported promising results. However, some have questioned the effectiveness of using ImageNet, which consists solely of natural images, for medical image analysis. The aim of this study was to evaluate whether pre-trained models using RadImageNet, which is a large-scale medical image dataset, could achieve superior performance in classification tasks in dental imaging modalities compared with ImageNet pre-trained models. To evaluate the classification performance of RadImageNet and ImageNet pre-trained models for TL, two dental imaging datasets were used. The tasks were (1) classifying the presence or absence of supernumerary teeth from a dataset of panoramic radiographs and (2) classifying sex from a dataset of lateral cephalometric radiographs. Performance was evaluated by comparing the area under the curve (AUC). On the panoramic radiograph dataset, the RadImageNet models gave average AUCs of 0.68 ± 0.15 (p < 0.01), and the ImageNet models had values of 0.74 ± 0.19. In contrast, on the lateral cephalometric dataset, the RadImageNet models demonstrated average AUCs of 0.76 ± 0.09, and the ImageNet models achieved values of 0.75 ± 0.17. The difference in performance between RadImageNet and ImageNet models in TL depends on the dental image dataset used.

Selection of pre-trained weights for transfer learning in automated cytomegalovirus retinitis classification

Cytomegalovirus retinitis (CMVR) is a significant cause of vision loss. Regular screening is crucial but challenging in resource-limited settings. A convolutional neural network is a state-of-the-art deep learning technique to generate automatic diagnoses from retinal images. However, there are limited numbers of CMVR images to train the model properly. Transfer learning (TL) is a strategy to train a model with a scarce dataset. This study explores the efficacy of TL with different pre-trained weights for automated CMVR classification using retinal images. We utilised a dataset of 955 retinal images (524 CMVR and 431 normal) from Siriraj Hospital, Mahidol University, collected between 2005 and 2015. Images were processed using Kowa VX-10i or VX-20 fundus cameras and augmented for training. We employed DenseNet121 as a backbone model, comparing the performance of TL with weights pre-trained on ImageNet, APTOS2019, and CheXNet datasets. The models were evaluated based on accuracy, loss, and other performance metrics, with the depth of fine-tuning varied across different pre-trained weights. The study found that TL significantly enhances model performance in CMVR classification. The best results were achieved with weights sequentially transferred from ImageNet to APTOS2019 dataset before application to our CMVR dataset. This approach yielded the highest mean accuracy (0.99) and lowest mean loss (0.04), outperforming other methods. The class activation heatmaps provided insights into the model's decision-making process. The model with APTOS2019 pre-trained weights offered the best explanation and highlighted the pathologic lesions resembling human interpretation. Our findings demonstrate the potential of sequential TL in improving the accuracy and efficiency of CMVR diagnosis, particularly in settings with limited data availability. They highlight the importance of domain-specific pre-training in medical image classification. This approach streamlines the diagnostic process and paves the way for broader applications in automated medical image analysis, offering a scalable solution for early disease detection.

Multi-task neural networks by learned contextual inputs

This paper explores learned-context neural networks. It is a multi-task learning architecture based on a fully shared neural network and an augmented input vector containing trainable task parameters. The architecture is interesting due to its powerful task adaption mechanism, which facilitates a low-dimensional task parameter space. Theoretically, we show that a scalar task parameter is sufficient for universal approximation of all tasks, which is not necessarily the case for more common architectures. Empirically it is shown that, for homogeneous tasks, the dimension of the task parameter may vary with the complexity of the tasks, but a small task parameter space is generally viable. The task parameter space is found to be well-behaved, which simplifies workflows related to updating models as new data arrives, and learning new tasks with the shared parameters are frozen. Additionally, the architecture displays robustness towards datasets where tasks have few data points. The architecture's performance is compared to similar neural network architectures on ten datasets, with competitive results.

Comparison of deep learning architectures for predicting amyloid positivity in Alzheimer's disease, mild cognitive impairment, and healthy aging, from T1-weighted brain structural MRI

Abnormal β-amyloid (Aβ) accumulation in the brain is an early indicator of Alzheimer's disease (AD) and is typically assessed through invasive procedures such as PET (positron emission tomography) or CSF (cerebrospinal fluid) assays. As new anti-Alzheimer's treatments can now successfully target amyloid pathology, there is a growing interest in predicting Aβ positivity (Aβ+) from less invasive, more widely available types of brain scans, such as T1-weighted (T1w) MRI. Here we compare multiple approaches to infer Aβ + from standard anatomical MRI: (1) classical machine learning algorithms, including logistic regression, XGBoost, and shallow artificial neural networks, (2) deep learning models based on 2D and 3D convolutional neural networks (CNNs), (3) a hybrid ANN-CNN, combining the strengths of shallow and deep neural networks, (4) transfer learning models based on CNNs, and (5) 3D Vision Transformers. All models were trained on paired MRI/PET data from 1,847 elderly participants (mean age: 75.1 yrs. ± 7.6SD; 863 females/984 males; 661 healthy controls, 889 with mild cognitive impairment (MCI), and 297 with Dementia), scanned as part of the Alzheimer's Disease Neuroimaging Initiative. We evaluated each model's balanced accuracy and F1 scores. While further tests on more diverse data are warranted, deep learning models trained on standard MRI showed promise for estimating Aβ + status, at least in people with MCI. This may offer a potential screening option before resorting to more invasive procedures.

Image Quality Assessment Using Convolutional Neural Network in Clinical Skin Images

The image quality received for clinical evaluation is often suboptimal. The goal is to develop an image quality analysis tool to assess patient- and primary care physician-derived images using deep learning model. Dataset included patient- and primary care physician-derived images from August 21, 2018 to June 30, 2022 with 4 unique quality labels. VGG16 model was fine tuned with input data, and optimal threshold was determined by Youden's index. Ordinal labels were transformed to binary labels using a majority vote because model distinguishes between 2 categories (good vs bad). At a threshold of 0.587, area under the curve for the test set was 0.885 (95% confidence interval = 0.838-0.933); sensitivity, specificity, positive predictive value, and negative predictive value were 0.829, 0.784, 0.906, and 0.645, respectively. Independent validation of 300 additional images (from patients and primary care physicians) demonstrated area under the curve of 0.864 (95% confidence interval = 0.818-0.909) and area under the curve of 0.902 (95% confidence interval = 0.85-0.95), respectively. The sensitivity, specificity, positive predictive value, and negative predictive value for the 300 images were 0.827, 0.800, 0.959, and 0.450, respectively. We demonstrate a practical approach improving the image quality for clinical workflow. Although users may have to capture additional images, this is offset by the improved workload and efficiency for clinical teams.

Augmenting Radiological Diagnostics with AI for Tuberculosis and COVID-19 Disease Detection: Deep Learning Detection of Chest Radiographs

In this research, we introduce a network that can identify pneumonia, COVID-19, and tuberculosis using X-ray images of patients' chests. The study emphasizes tuberculosis, COVID-19, and healthy lung conditions, discussing how advanced neural networks, like VGG16 and ResNet50, can improve the detection of lung issues from images. To prepare the images for the model's input requirements, we enhanced them through data augmentation techniques for training purposes. We evaluated the model's performance by analyzing the precision, recall, and F1 scores across training, validation, and testing datasets. The results show that the ResNet50 model outperformed VGG16 with accuracy and resilience. It displayed superior ROC AUC values in both validation and test scenarios. Particularly impressive were ResNet50's precision and recall rates, nearing 0.99 for all conditions in the test set. On the hand, VGG16 also performed well during testing-detecting tuberculosis with a precision of 0.99 and a recall of 0.93. Our study highlights the performance of our deep learning method by showcasing the effectiveness of ResNet50 over traditional approaches like VGG16. This progress utilizes methods to enhance classification accuracy by augmenting data and balancing them. This positions our approach as an advancement in using state-of-the-art deep learning applications in imaging. By enhancing the accuracy and reliability of diagnosing ailments such as COVID-19 and tuberculosis, our models have the potential to transform care and treatment strategies, highlighting their role in clinical diagnostics.

Sex estimation from maxillofacial radiographs using a deep learning approach

The purpose of this study was to construct deep learning models for more efficient and reliable sex estimation. Two deep learning models, VGG16 and DenseNet-121, were used in this retrospective study. In total, 600 lateral cephalograms were analyzed. A saliency map was generated by gradient-weighted class activation mapping for each output. The two deep learning models achieved high values in each performance metric according to accuracy, sensitivity (recall), precision, F1 score, and areas under the receiver operating characteristic curve. Both models showed substantial differences in the positions indicated in saliency maps for male and female images. The positions in saliency maps also differed between VGG16 and DenseNet-121, regardless of sex. This analysis of our proposed system suggested that sex estimation from lateral cephalograms can be achieved with high accuracy using deep learning.

ImSpect: Image-driven self-supervised learning for surgical margin evaluation with mass spectrometry

PURPOSE

Real-time assessment of surgical margins is critical for favorable outcomes in cancer patients. The iKnife is a mass spectrometry device that has demonstrated potential for margin detection in cancer surgery. Previous studies have shown that using deep learning on iKnife data can facilitate real-time tissue characterization. However, none of the existing literature on the iKnife facilitate the use of publicly available, state-of-the-art pretrained networks or datasets that have been used in computer vision and other domains.

METHODS

In a new framework we call ImSpect, we convert 1D iKnife data, captured during basal cell carcinoma (BCC) surgery, into 2D images in order to capitalize on state-of-the-art image classification networks. We also use self-supervision to leverage large amounts of unlabeled, intraoperative data to accommodate the data requirements of these networks.

RESULTS

Through extensive ablation studies, we show that we can surpass previous benchmarks of margin evaluation in BCC surgery using iKnife data, achieving an area under the receiver operating characteristic curve (AUC) of 81%. We also depict the attention maps of the developed DL models to evaluate the biological relevance of the embedding space CONCLUSIONS: We propose a new method for characterizing tissue at the surgical margins, using mass spectrometry data from cancer surgery.

Semi-supervised model based on implicit neural representation and mutual learning (SIMN) for multi-center nasopharyngeal carcinoma segmentation on MRI

BACKGROUND

The issue of using deep learning to obtain accurate gross tumor volume (GTV) and metastatic lymph nodes (MLN) segmentation for nasopharyngeal carcinoma (NPC) on heterogeneous magnetic resonance imaging (MRI) images with limited labeling remains unsolved.

METHOD

We collected 918 patients with MRI images from three hospitals to develop and validate models and proposed a semi-supervised framework for the fine delineation of multi-center NPC boundaries by integrating uncertainty-based implicit neural representations named SIMN. The framework utilizes the deep mutual learning approach with CNN and Transformer, incorporating dynamic thresholds. Additionally, domain adaptive algorithms are employed to enhance the performance.

RESULTS

SIMN predictions have a high overlap ratio with the ground truth. Under the 20 % labeled cases, for the internal test cohorts, the average DSC in GTV and MLN are 0.7981 and 0.7804, respectively; for external test cohort Wu Zhou Red Cross Hospital, the average DSC in GTV and MLN are 0.7217 and 0.7581, respectively; for external test cohorts First People Hospital of Foshan, the average DSC in GTV and MLN are 0.7004 and 0.7692, respectively. No significant differences are found in DSC, HD95, ASD, and Recall for patients with different clinical categories. Moreover, SIMN outperformed existing classical semi-supervised methods.

CONCLUSIONS

SIMN showed a highly accurate GTV and MLN segmentation for NPC on multi-center MRI images under Semi-Supervised Learning (SSL), which can easily transfer to other centers without fine-tuning. It suggests that it has the potential to act as a generalized delineation solution for heterogeneous MRI images with limited labels in clinical deployment.

BarlowTwins-CXR: enhancing chest X-ray abnormality localization in heterogeneous data with cross-domain self-supervised learning

BACKGROUND

Chest X-ray imaging based abnormality localization, essential in diagnosing various diseases, faces significant clinical challenges due to complex interpretations and the growing workload of radiologists. While recent advances in deep learning offer promising solutions, there is still a critical issue of domain inconsistency in cross-domain transfer learning, which hampers the efficiency and accuracy of diagnostic processes. This study aims to address the domain inconsistency problem and improve autonomic abnormality localization performance of heterogeneous chest X-ray image analysis, particularly in detecting abnormalities, by developing a self-supervised learning strategy called "BarlwoTwins-CXR".

METHODS

We utilized two publicly available datasets: the NIH Chest X-ray Dataset and the VinDr-CXR. The BarlowTwins-CXR approach was conducted in a two-stage training process. Initially, self-supervised pre-training was performed using an adjusted Barlow Twins algorithm on the NIH dataset with a Resnet50 backbone pre-trained on ImageNet. This was followed by supervised fine-tuning on the VinDr-CXR dataset using Faster R-CNN with Feature Pyramid Network (FPN). The study employed mean Average Precision (mAP) at an Intersection over Union (IoU) of 50% and Area Under the Curve (AUC) for performance evaluation.

RESULTS

Our experiments showed a significant improvement in model performance with BarlowTwins-CXR. The approach achieved a 3% increase in mAP50 accuracy compared to traditional ImageNet pre-trained models. In addition, the Ablation CAM method revealed enhanced precision in localizing chest abnormalities. The study involved 112,120 images from the NIH dataset and 18,000 images from the VinDr-CXR dataset, indicating robust training and testing samples.

CONCLUSION

BarlowTwins-CXR significantly enhances the efficiency and accuracy of chest X-ray image-based abnormality localization, outperforming traditional transfer learning methods and effectively overcoming domain inconsistency in cross-domain scenarios. Our experiment results demonstrate the potential of using self-supervised learning to improve the generalizability of models in medical settings with limited amounts of heterogeneous data. This approach can be instrumental in aiding radiologists, particularly in high-workload environments, offering a promising direction for future AI-driven healthcare solutions.

Convolutional Neural Network to Classify Infrared Thermal Images of Fractured Wrists in Pediatrics

Convolutional neural network (CNN) models were devised and evaluated to classify infrared thermal (IRT) images of pediatric wrist fractures. The images were recorded from 19 participants with a wrist fracture and 21 without a fracture (sprain). The injury diagnosis was by X-ray radiography. For each participant, 299 IRT images of their wrists were recorded. These generated 11,960 images (40 participants × 299 images). For each image, the wrist region of interest (ROI) was selected and fast Fourier transformed (FFT) to obtain a magnitude frequency spectrum. The spectrum was resized to 100 × 100 pixels from its center as this region represented the main frequency components. Image augmentations of rotation, translation and shearing were applied to the 11,960 magnitude frequency spectra to assist with the CNN generalization during training. The CNN had 34 layers associated with convolution, batch normalization, rectified linear unit, maximum pooling and SoftMax and classification. The ratio of images for the training and test was 70:30, respectively. The effects of augmentation and dropout on CNN performance were explored. Wrist fracture identification sensitivity and accuracy of 88% and 76%, respectively, were achieved. The CNN model was able to identify wrist fractures; however, a larger sample size would improve accuracy.

OPG-based dental age estimation using a data-technical exploration of deep learning techniques

Dental age estimation, a cornerstone in forensic age assessment, has been extensively tried and tested, yet manual methods are impeded by tedium and interobserver variability. Automated approaches using deep transfer learning encounter challenges like data scarcity, suboptimal training, and fine-tuning complexities, necessitating robust training methods. This study explores the impact of convolutional neural network hyperparameters, model complexity, training batch size, and sample quantity on age estimation. EfficientNet-B4, DenseNet-201, and MobileNet V3 models underwent cross-validation on a dataset of 3896 orthopantomograms (OPGs) with batch sizes escalating from 10 to 160 in a doubling progression, as well as random subsets of this training dataset. Results demonstrate the EfficientNet-B4 model, trained on the complete dataset with a batch size of 160, as the top performer with a mean absolute error of 0.562 years on the test set, notably surpassing the MAE of 1.01 at a batch size of 10. Increasing batch size consistently improved performance for EfficientNet-B4 and DenseNet-201, whereas MobileNet V3 performance peaked at batch size 40. Similar trends emerged in training with reduced sample sizes, though they were outperformed by the complete models. This underscores the critical role of hyperparameter optimization in adopting deep learning for age estimation from complete OPGs. The findings not only highlight the nuanced interplay of hyperparameters and performance but also underscore the potential for accurate age estimation models through optimization. This study contributes to advancing the application of deep learning in forensic age estimation, emphasizing the significance of tailored training methodologies for optimal outcomes.

Detecting dental caries on oral photographs using artificial intelligence: A systematic review

OBJECTIVES

This systematic review aimed at evaluating the performance of artificial intelligence (AI) models in detecting dental caries on oral photographs.

METHODS

Methodological characteristics and performance metrics of clinical studies reporting on deep learning and other machine learning algorithms were assessed. The risk of bias was evaluated using the quality assessment of diagnostic accuracy studies 2 (QUADAS-2) tool. A systematic search was conducted in EMBASE, Medline, and Scopus.

RESULTS

Out of 3410 identified records, 19 studies were included with six and seven studies having low risk of biases and applicability concerns for all the domains, respectively. Metrics varied widely and were assessed on multiple levels. F1-scores for classification and detection tasks were 68.3%-94.3% and 42.8%-95.4%, respectively. Irrespective of the task, F1-scores were 68.3%-95.4% for professional cameras, 78.8%-87.6%, for intraoral cameras, and 42.8%-80% for smartphone cameras. Limited studies allowed assessing AI performance for lesions of different severity.

CONCLUSION

Automatic detection of dental caries using AI may provide objective verification of clinicians' diagnoses and facilitate patient-clinician communication and teledentistry. Future studies should consider more robust study designs, employ comparable and standardized metrics, and focus on the severity of caries lesions.

Automated detection of otosclerosis with interpretable deep learning using temporal bone computed tomography images

Objective

This study aimed to develop an automated detection schema for otosclerosis with interpretable deep learning using temporal bone computed tomography images.

Methods

With approval from the institutional review board, we retrospectively analyzed high-resolution computed tomography scans of the temporal bone of 182 participants with otosclerosis (67 male subjects and 115 female subjects; average age, 36.42 years) and 157 participants without otosclerosis (52 male subjects and 102 female subjects; average age, 30.61 years) using deep learning. Transfer learning with the pretrained VGG19, Mask RCNN, and EfficientNet models was used. In addition, 3 clinical experts compared the system's performance by reading the same computed tomography images for a subset of 35 unseen subjects. An area under the receiver operating characteristic curve and a saliency map were used to further evaluate the diagnostic performance.

Results

In prospective unseen test data, the diagnostic performance of the automatically interpretable otosclerosis detection system at the optimal threshold was 0.97 and 0.98 for sensitivity and specificity, respectively. In comparison with the clinical acumen of otolaryngologists at P < 0.05, the proposed system was not significantly different. Moreover, the area under the receiver operating characteristic curve for the proposed system was 0.99, indicating satisfactory diagnostic accuracy.

Conclusion

Our research develops and evaluates a deep learning system that detects otosclerosis at a level comparable with clinical otolaryngologists. Our system is an effective schema for the differential diagnosis of otosclerosis in computed tomography examinations.

Medical image foundation models in assisting diagnosis of brain tumors: a pilot study

OBJECTIVES

To build self-supervised foundation models for multicontrast MRI of the whole brain and evaluate their efficacy in assisting diagnosis of brain tumors.

METHODS

In this retrospective study, foundation models were developed using 57,621 enhanced head MRI scans through self-supervised learning with a pretext task of cross-contrast context restoration with two different content dropout schemes. Downstream classifiers were constructed based on the pretrained foundation models and fine-tuned for brain tumor detection, discrimination, and molecular status prediction. Metrics including accuracy, sensitivity, specificity, and area under the ROC curve (AUC) were used to evaluate the performance. Convolutional neural networks trained exclusively on downstream task data were employed for comparative analysis.

RESULTS

The pretrained foundation models demonstrated their ability to extract effective representations from multicontrast whole-brain volumes. The best classifiers, endowed with pretrained weights, showed remarkable performance with accuracies of 94.9, 92.3, and 80.4%, and corresponding AUC values of 0.981, 0.972, and 0.852 on independent test datasets in brain tumor detection, discrimination, and molecular status prediction, respectively. The classifiers with pretrained weights outperformed the convolutional classifiers trained from scratch by approximately 10% in terms of accuracy and AUC across all tasks. The saliency regions in the correctly predicted cases are mainly clustered around the tumors. Classifiers derived from the two dropout schemes differed significantly only in the detection of brain tumors.

CONCLUSIONS

Foundation models obtained from self-supervised learning have demonstrated encouraging potential for scalability and interpretability in downstream brain tumor-related tasks and hold promise for extension to neurological diseases with diffusely distributed lesions.

CLINICAL RELEVANCE STATEMENT

The application of our proposed method to the prediction of key molecular status in gliomas is expected to improve treatment planning and patient outcomes. Additionally, the foundation model we developed could serve as a cornerstone for advancing AI applications in the diagnosis of brain-related diseases.

Rule-based deep learning method for prognosis of neonatal hypoxic-ischemic encephalopathy by using susceptibility weighted image analysis

OBJECTIVE

Susceptibility weighted imaging (SWI) of neonatal hypoxic-ischemic brain injury can provide assistance in the prognosis of neonatal hypoxic-ischemic encephalopathy (HIE). We propose a convolutional neural network model to classify SWI images with HIE.

MATERIALS AND METHODS

Due to the lack of a large dataset, transfer learning method with fine-tuning a pre-trained ResNet 50 is introduced. We randomly select 11 datasets from patients with normal neurology outcomes (n = 31) and patients with abnormal neurology outcomes (n = 11) at 24 months of age to avoid bias in classification due to any imbalance in the data.

RESULTS

We develop a rule-based system to improve the classification performance, with an accuracy of 0.93 ± 0.09. We also compute heatmaps produced by the Grad-CAM technique to analyze which areas of SWI images contributed more to the classification patients with abnormal neurology outcome.

CONCLUSION

Such regions that are important in the classification accuracy can interpret the relationship between the brain regions affected by hypoxic-ischemic and neurodevelopmental outcomes of infants with HIE at the age of 2 years.

Transfer learning for the generalization of artificial intelligence in breast cancer detection: a case-control study

BACKGROUND

Some researchers have questioned whether artificial intelligence (AI) systems maintain their performance when used for women from populations not considered during the development of the system.

PURPOSE

To evaluate the impact of transfer learning as a way of improving the generalization of AI systems in the detection of breast cancer.

MATERIAL AND METHODS

This retrospective case-control Finnish study involved 191 women diagnosed with breast cancer and 191 matched healthy controls. We selected a state-of-the-art AI system for breast cancer detection trained using a large US dataset. The selected baseline system was evaluated in two experimental settings. First, we examined our private Finnish sample as an independent test set that had not been considered in the development of the system (unseen population). Second, the baseline system was retrained to attempt to improve its performance in the unseen population by means of transfer learning. To analyze performance, we used areas under the receiver operating characteristic curve (AUCs) with DeLong's test.

RESULTS

Two versions of the baseline system were considered: ImageOnly and Heatmaps. The ImageOnly and Heatmaps versions yielded mean AUC values of 0.82±0.008 and 0.88±0.003 in the US dataset and 0.56 (95% CI=0.50-0.62) and 0.72 (95% CI=0.67-0.77) when evaluated in the unseen population, respectively. The retrained systems achieved AUC values of 0.61 (95% CI=0.55-0.66) and 0.69 (95% CI=0.64-0.75), respectively. There was no statistical difference between the baseline system and the retrained system.

CONCLUSION

Transfer learning with a small study sample did not yield a significant improvement in the generalization of the system.

Risk prediction of pulse wave for hypertensive target organ damage based on frequency-domain feature map

The application of deep learning to the classification of pulse waves in Traditional Chinese Medicine (TCM) related to hypertensive target organ damage (TOD) is hindered by challenges such as low classification accuracy and inadequate generalization performance. To address these challenges, we introduce a lightweight transfer learning model named MobileNetV2SCP. This model transforms time-domain pulse waves into 36-dimensional frequency-domain waveform feature maps and establishes a dedicated pre-training network based on these maps to enhance the learning capability for small samples. To improve global feature correlation, we incorporate a novel fusion attention mechanism (SAS) into the inverted residual structure, along with the utilization of 3 × 3 convolutional layers and BatchNorm layers to mitigate model overfitting. The proposed model is evaluated using cross-validation results from 805 cases of pulse waves associated with hypertensive TOD. The assessment metrics, including Accuracy (92.74 %), F1-score (91.47 %), and Area Under Curve (AUC) (97.12 %), demonstrate superior classification accuracy and generalization performance compared to various state-of-the-art models. Furthermore, this study investigates the correlations between time-domain and frequency-domain features in pulse waves and their classification in hypertensive TOD. It analyzes key factors influencing pulse wave classification, providing valuable insights for the clinical diagnosis of TOD.

Feature extraction of particle morphologies of pharmaceutical excipients from scanning electron microscope images using convolutional neural networks

Scanning electron microscopy (SEM) images are the most widely used tool for evaluating particle morphology; however, quantitative evaluation using SEM images is time-consuming and often neglected. In this study, we aimed to extract features related to particle morphology of pharmaceutical excipients from SEM images using a convolutional neural network (CNN). SEM images of 67 excipients were acquired and used as models. A classification CNN model of the excipients was constructed based on the SEM images. Further, features were extracted from the middle layer of this CNN model, and the data was compressed to two dimensions using uniform manifold approximation and projection. Lastly, hierarchical clustering analysis (HCA) was performed to categorize the excipients into several clusters and identify similarities among the samples. The classification CNN model showed high accuracy, allowing each excipient to be identified with a high degree of accuracy. HCA revealed that the 67 excipients were classified into seven clusters. Additionally, the particle morphologies of excipients belonging to the same cluster were found to be very similar. These results suggest that CNN models are useful tools for extracting information and identifying similarities among the particle morphologies of excipients.

Contrast-enhanced to non-contrast-enhanced image translation to exploit a clinical data warehouse of T1-weighted brain MRI

BACKGROUND

Clinical data warehouses provide access to massive amounts of medical images, but these images are often heterogeneous. They can for instance include images acquired both with or without the injection of a gadolinium-based contrast agent. Harmonizing such data sets is thus fundamental to guarantee unbiased results, for example when performing differential diagnosis. Furthermore, classical neuroimaging software tools for feature extraction are typically applied only to images without gadolinium. The objective of this work is to evaluate how image translation can be useful to exploit a highly heterogeneous data set containing both contrast-enhanced and non-contrast-enhanced images from a clinical data warehouse.

METHODS

We propose and compare different 3D U-Net and conditional GAN models to convert contrast-enhanced T1-weighted (T1ce) into non-contrast-enhanced (T1nce) brain MRI. These models were trained using 230 image pairs and tested on 77 image pairs from the clinical data warehouse of the Greater Paris area.

RESULTS

Validation using standard image similarity measures demonstrated that the similarity between real and synthetic T1nce images was higher than between real T1nce and T1ce images for all the models compared. The best performing models were further validated on a segmentation task. We showed that tissue volumes extracted from synthetic T1nce images were closer to those of real T1nce images than volumes extracted from T1ce images.

CONCLUSION

We showed that deep learning models initially developed with research quality data could synthesize T1nce from T1ce images of clinical quality and that reliable features could be extracted from the synthetic images, thus demonstrating the ability of such methods to help exploit a data set coming from a clinical data warehouse.

A comprehensive evaluation of large Language models on benchmark biomedical text processing tasks

Recently, Large Language Models (LLMs) have demonstrated impressive capability to solve a wide range of tasks. However, despite their success across various tasks, no prior work has investigated their capability in the biomedical domain yet. To this end, this paper aims to evaluate the performance of LLMs on benchmark biomedical tasks. For this purpose, a comprehensive evaluation of 4 popular LLMs in 6 diverse biomedical tasks across 26 datasets has been conducted. To the best of our knowledge, this is the first work that conducts an extensive evaluation and comparison of various LLMs in the biomedical domain. Interestingly, we find based on our evaluation that in biomedical datasets that have smaller training sets, zero-shot LLMs even outperform the current state-of-the-art models when they were fine-tuned only on the training set of these datasets. This suggests that pre-training on large text corpora makes LLMs quite specialized even in the biomedical domain. We also find that not a single LLM can outperform other LLMs in all tasks, with the performance of different LLMs may vary depending on the task. While their performance is still quite poor in comparison to the biomedical models that were fine-tuned on large training sets, our findings demonstrate that LLMs have the potential to be a valuable tool for various biomedical tasks that lack large annotated data.

Artificial intelligence for radiographic imaging detection of caries lesions: a systematic review

BACKGROUND

The aim of this systematic review is to evaluate the diagnostic performance of Artificial Intelligence (AI) models designed for the detection of caries lesion (CL).

MATERIALS AND METHODS

An electronic literature search was conducted on PubMed, Web of Science, SCOPUS, LILACS and Embase databases for retrospective, prospective and cross-sectional studies published until January 2023, using the following keywords: artificial intelligence (AI), machine learning (ML), deep learning (DL), artificial neural networks (ANN), convolutional neural networks (CNN), deep convolutional neural networks (DCNN), radiology, detection, diagnosis and dental caries (DC). The quality assessment was performed using the guidelines of QUADAS-2.

RESULTS

Twenty articles that met the selection criteria were evaluated. Five studies were performed on periapical radiographs, nine on bitewings, and six on orthopantomography. The number of imaging examinations included ranged from 15 to 2900. Four studies investigated ANN models, fifteen CNN models, and two DCNN models. Twelve were retrospective studies, six cross-sectional and two prospective. The following diagnostic performance was achieved in detecting CL: sensitivity from 0.44 to 0.86, specificity from 0.85 to 0.98, precision from 0.50 to 0.94, PPV (Positive Predictive Value) 0.86, NPV (Negative Predictive Value) 0.95, accuracy from 0.73 to 0.98, area under the curve (AUC) from 0.84 to 0.98, intersection over union of 0.3-0.4 and 0.78, Dice coefficient 0.66 and 0.88, F1-score from 0.64 to 0.92. According to the QUADAS-2 evaluation, most studies exhibited a low risk of bias.

CONCLUSION

AI-based models have demonstrated good diagnostic performance, potentially being an important aid in CL detection. Some limitations of these studies are related to the size and heterogeneity of the datasets. Future studies need to rely on comparable, large, and clinically meaningful datasets.

PROTOCOL

PROSPERO identifier: CRD42023470708.

Application of deep learning for automated diagnosis and classification of hip dysplasia on plain radiographs

BACKGROUND

Hip dysplasia is a condition where the acetabulum is too shallow to support the femoral head and is commonly considered a risk factor for hip osteoarthritis. The objective of this study was to develop a deep learning model to diagnose hip dysplasia from plain radiographs and classify dysplastic hips based on their severity.

METHODS

We collected pelvic radiographs of 571 patients from two single-center cohorts and one multicenter cohort. The radiographs were split in half to create hip radiographs (n = 1022). One orthopaedic surgeon and one resident assessed the radiographs for hip dysplasia on either side. We used the center edge (CE) angle as the primary diagnostic criteria. Hips with a CE angle < 20°, 20° to 25°, and > 25° were labeled as dysplastic, borderline, and normal, respectively. The dysplastic hips were also classified with both Crowe and Hartofilakidis classification of dysplasia. The dataset was divided into train, validation, and test subsets using 80:10:10 split-ratio that were used to train two deep learning models to classify images into normal, borderline and (1) Crowe grade 1-4 or (2) Hartofilakidis grade 1-3. A pre-trained on Imagenet VGG16 convolutional neural network (CNN) was utilized by performing layer-wise fine-turning.

RESULTS

Both models struggled with distinguishing between normal and borderline hips. However, achieved high accuracy (Model 1: 92.2% and Model 2: 83.3%) in distinguishing between normal/borderline vs. dysplastic hips. The overall accuracy of Model 1 was 68% and for Model 2 73.5%. Most misclassifications for the Crowe and Hartofilakidis classifications were +/- 1 class from the correct class.

CONCLUSIONS

This pilot study shows promising results that a deep learning model distinguish between normal and dysplastic hips with high accuracy. Future research and external validation are warranted regarding the ability of deep learning models to perform complex tasks such as identifying and classifying disorders using plain radiographs.

LEVEL OF EVIDENCE

Diagnostic level IV.

Data pyramid structure for optimizing EUS-based GISTs diagnosis in multi-center analysis with missing label

This study introduces the Data Pyramid Structure (DPS) to address data sparsity and missing labels in medical image analysis. The DPS optimizes multi-task learning and enables sustainable expansion of multi-center data analysis. Specifically, It facilitates attribute prediction and malignant tumor diagnosis tasks by implementing a segmentation and aggregation strategy on data with absent attribute labels. To leverage multi-center data, we propose the Unified Ensemble Learning Framework (UELF) and the Unified Federated Learning Framework (UFLF), which incorporate strategies for data transfer and incremental learning in scenarios with missing labels. The proposed method was evaluated on a challenging EUS patient dataset from five centers, achieving promising diagnostic performance. The average accuracy was 0.984 with an AUC of 0.927 for multi-center analysis, surpassing state-of-the-art approaches. The interpretability of the predictions further highlights the potential clinical relevance of our method.

Machine-learned wearable sensors for real-time hand-motion recognition: toward practical applications

Soft electromechanical sensors have led to a new paradigm of electronic devices for novel motion-based wearable applications in our daily lives. However, the vast amount of random and unidentified signals generated by complex body motions has hindered the precise recognition and practical application of this technology. Recent advancements in artificial-intelligence technology have enabled significant strides in extracting features from massive and intricate data sets, thereby presenting a breakthrough in utilizing wearable sensors for practical applications. Beyond traditional machine-learning techniques for classifying simple gestures, advanced machine-learning algorithms have been developed to handle more complex and nuanced motion-based tasks with restricted training data sets. Machine-learning techniques have improved the ability to perceive, and thus machine-learned wearable soft sensors have enabled accurate and rapid human-gesture recognition, providing real-time feedback to users. This forms a crucial component of future wearable electronics, contributing to a robust human-machine interface. In this review, we provide a comprehensive summary covering materials, structures and machine-learning algorithms for hand-gesture recognition and possible practical applications through machine-learned wearable electromechanical sensors.

Generative deep learning furthers the understanding of local distributions of fat and muscle on body shape and health using 3D surface scans

BACKGROUND

Body shape, an intuitive health indicator, is deterministically driven by body composition. We developed and validated a deep learning model that generates accurate dual-energy X-ray absorptiometry (DXA) scans from three-dimensional optical body scans (3DO), enabling compositional analysis of the whole body and specified subregions. Previous works on generative medical imaging models lack quantitative validation and only report quality metrics.

METHODS

Our model was self-supervised pretrained on two large clinical DXA datasets and fine-tuned using the Shape Up! Adults study dataset. Model-predicted scans from a holdout test set were evaluated using clinical commercial DXA software for compositional accuracy.

RESULTS

Predicted DXA scans achieve R2 of 0.73, 0.89, and 0.99 and RMSEs of 5.32, 6.56, and 4.15 kg for total fat mass (FM), fat-free mass (FFM), and total mass, respectively. Custom subregion analysis results in R2s of 0.70-0.89 for left and right thigh composition. We demonstrate the ability of models to produce quantitatively accurate visualizations of soft tissue and bone, confirming a strong relationship between body shape and composition.

CONCLUSIONS

This work highlights the potential of generative models in medical imaging and reinforces the importance of quantitative validation for assessing their clinical utility.

A time series driven model for early sepsis prediction based on transformer module

Sepsis remains a critical concern in intensive care units due to its high mortality rate. Early identification and intervention are paramount to improving patient outcomes. In this study, we have proposed predictive models for early sepsis prediction based on time-series data, utilizing both CNN-Transformer and LSTM-Transformer architectures. By collecting time-series data from patients at 4, 8, and 12 h prior to sepsis diagnosis and subjecting it to various network models for analysis and comparison. In contrast to traditional recurrent neural networks, our model exhibited a substantial improvement of approximately 20%. On average, our model demonstrated an accuracy of 0.964 (± 0.018), a precision of 0.956 (± 0.012), a recall of 0.967 (± 0.012), and an F1 score of 0.959 (± 0.014). Furthermore, by adjusting the time window, it was observed that the Transformer-based model demonstrated exceptional predictive capabilities, particularly within the earlier time window (i.e., 12 h before onset), thus holding significant promise for early clinical diagnosis and intervention. Besides, we employed the SHAP algorithm to visualize the weight distribution of different features, enhancing the interpretability of our model and facilitating early clinical diagnosis and intervention.

Phase retrieval for X-ray differential phase contrast radiography with knowledge transfer learning from virtual differential absorption model

Grating-based X-ray phase contrast radiography and computed tomography (CT) are promising modalities for future medical applications. However, the ill-posed phase retrieval problem in X-ray phase contrast imaging has hindered its use for quantitative analysis in biomedical imaging. Deep learning has been proved as an effective tool for image retrieval. However, in practical grating-based X-ray phase contrast imaging system, acquiring the ground truth of phase to form image pairs is challenging, which poses a great obstacle for using deep leaning methods. Transfer learning is widely used to address the problem with knowledge inheritance from similar tasks. In the present research, we propose a virtual differential absorption model and generate a training dataset with differential absorption images and absorption images. The knowledge learned from the training is transferred to phase retrieval with transfer learning techniques. Numerical simulations and experiments both demonstrate its feasibility. Image quality of retrieved phase radiograph and phase CT slices is improved when compared with representative phase retrieval methods. We conclude that this method is helpful in both X-ray 2D and 3D imaging and may find its applications in X-ray phase contrast radiography and X-ray phase CT.

Combining State-of-the-Art Pre-Trained Deep Learning Models: A Noble Approach for Skin Cancer Detection Using Max Voting Ensemble

Skin cancer poses a significant healthcare challenge, requiring precise and prompt diagnosis for effective treatment. While recent advances in deep learning have dramatically improved medical image analysis, including skin cancer classification, ensemble methods offer a pathway for further enhancing diagnostic accuracy. This study introduces a cutting-edge approach employing the Max Voting Ensemble Technique for robust skin cancer classification on ISIC 2018: Task 1-2 dataset. We incorporate a range of cutting-edge, pre-trained deep neural networks, including MobileNetV2, AlexNet, VGG16, ResNet50, DenseNet201, DenseNet121, InceptionV3, ResNet50V2, InceptionResNetV2, and Xception. These models have been extensively trained on skin cancer datasets, achieving individual accuracies ranging from 77.20% to 91.90%. Our method leverages the synergistic capabilities of these models by combining their complementary features to elevate classification performance further. In our approach, input images undergo preprocessing for model compatibility. The ensemble integrates the pre-trained models with their architectures and weights preserved. For each skin lesion image under examination, every model produces a prediction. These are subsequently aggregated using the max voting ensemble technique to yield the final classification, with the majority-voted class serving as the conclusive prediction. Through comprehensive testing on a diverse dataset, our ensemble outperformed individual models, attaining an accuracy of 93.18% and an AUC score of 0.9320, thus demonstrating superior diagnostic reliability and accuracy. We evaluated the effectiveness of our proposed method on the HAM10000 dataset to ensure its generalizability. Our ensemble method delivers a robust, reliable, and effective tool for the classification of skin cancer. By utilizing the power of advanced deep neural networks, we aim to assist healthcare professionals in achieving timely and accurate diagnoses, ultimately reducing mortality rates and enhancing patient outcomes.

Gastro-BaseNet: A Specialized Pre-Trained Model for Enhanced Gastroscopic Data Classification and Diagnosis of Gastric Cancer and Ulcer

Most of the development of gastric disease prediction models has utilized pre-trained models from natural data, such as ImageNet, which lack knowledge of medical domains. This study proposes Gastro-BaseNet, a classification model trained using gastroscopic image data for abnormal gastric lesions. To prove performance, we compared transfer-learning based on two pre-trained models (Gastro-BaseNet and ImageNet) and two training methods (freeze and fine-tune modes). The effectiveness was verified in terms of classification at the image-level and patient-level, as well as the localization performance of lesions. The development of Gastro-BaseNet had demonstrated superior transfer learning performance compared to random weight settings in ImageNet. When developing a model for predicting the diagnosis of gastric cancer and gastric ulcers, the transfer-learned model based on Gastro-BaseNet outperformed that based on ImageNet. Furthermore, the model's performance was highest when fine-tuning the entire layer in the fine-tune mode. Additionally, the trained model was based on Gastro-BaseNet, which showed higher localization performance, which confirmed its accurate detection and classification of lesions in specific locations. This study represents a notable advancement in the development of image analysis models within the medical field, resulting in improved diagnostic predictive accuracy and aiding in making more informed clinical decisions in gastrointestinal endoscopy.

The use of deep learning enables high diagnostic accuracy in detecting syndesmotic instability on weight-bearing CT scanning

PURPOSE

Delayed diagnosis of syndesmosis instability can lead to significant morbidity and accelerated arthritic change in the ankle joint. Weight-bearing computed tomography (WBCT) has shown promising potential for early and reliable detection of isolated syndesmotic instability using 3D volumetric measurements. While these measurements have been reported to be highly accurate, they are also experience-dependent, time-consuming, and need a particular 3D measurement software tool that leads the clinicians to still show more interest in the conventional diagnostic methods for syndesmotic instability. The purpose of this study was to increase accuracy, accelerate analysis time, and reduce interobserver bias by automating 3D volume assessment of syndesmosis anatomy using WBCT scans.

METHODS

A retrospective study was conducted using previously collected WBCT scans of patients with unilateral syndesmotic instability. One-hundred and forty-four bilateral ankle WBCT scans were evaluated (48 unstable, 96 control). We developed three deep learning models for analyzing WBCT scans to recognize syndesmosis instability. These three models included two state-of-the-art models (Model 1-3D Convolutional Neural Network [CNN], and Model 2-CNN with long short-term memory [LSTM]), and a new model (Model 3-differential CNN LSTM) that we introduced in this study.

RESULTS

Model 1 failed to analyze the WBCT scans (F1 score = 0). Model 2 only misclassified two cases (F1 score = 0.80). Model 3 outperformed Model 2 and achieved a nearly perfect performance, misclassifying only one case (F1 score = 0.91) in the control group as unstable while being faster than Model 2.

CONCLUSIONS

In this study, a deep learning model for 3D WBCT syndesmosis assessment was developed that achieved very high accuracy and accelerated analytics. This deep learning model shows promise for use by clinicians to improve diagnostic accuracy, reduce measurement bias, and save both time and expenditure for the healthcare system.

LEVEL OF EVIDENCE

II.

Can artificial intelligence help decision-making in arthroscopy? Part 2: The IA-RTRHO model - a decision-making aid for long head of the biceps diagnoses in small rotator cuff tears

INTRODUCTION

The possible applications of artificial intelligence (AI) in orthopedic surgery are promising. Deep learning can be utilized in arthroscopic surgery due to the video signal used by computer vision. The intraoperative management of the long head of biceps (LHB) tendon is the subject of a long-standing controversy. The main objective of this study was to model a diagnostic AI capable of determining the healthy or pathological state of the LHB on arthroscopic images. The secondary objective was to create a second diagnostic AI model based on arthroscopic images and the medical, clinical and imaging data of each patient, to determine the healthy or pathological state of the LHB.

HYPOTHESIS

The hypothesis of this study was that it was possible to construct an AI model from operative arthroscopic images to aid in the diagnosis of the healthy or pathological state of the LHB, and its analysis would be superior to a human analysis.

MATERIALS AND METHODS

Prospective clinical and imaging data from 199 patients were collected and associated with images from a validated protocoled arthroscopic video analysis, called "ground truth", made by the operating surgeon. A model based on a convolutional neural network (CNN) modeled via transfer learning on the Inception V3 model was built for the analysis of arthroscopic images. This model was then coupled to MultiLayer Perceptron (MLP), integrating clinical and imaging data. Each model was trained and tested using supervised learning.

RESULTS

The accuracy of the CNN in diagnosing the healthy or pathological state of the LHB was 93.7% in learning and 80.66% in generalization. Coupled with the clinical data of each patient, the accuracy of the model assembling the CNN and MLP were respectively 77% and 58% in learning and in generalization.

CONCLUSION

The AI model built from a CNN manages to determine the healthy or pathological state of the LHB with an accuracy rate of 80.66%. An increase in input data to limit overfitting, and the automation of the detection phase by a Mask-R-CNN are ways of improving the model. This study is the first to assess the ability of an AI to analyze arthroscopic images, and its results need to be confirmed by further studies on this subject.

LEVEL OF EVIDENCE

III Diagnostic study.

Designing a Deep Learning-Driven Resource-Efficient Diagnostic System for Metastatic Breast Cancer: Reducing Long Delays of Clinical Diagnosis and Improving Patient Survival in Developing Countries

Breast cancer is one of the leading causes of cancer mortality. Breast cancer patients in developing countries, especially sub-Saharan Africa, South Asia, and South America, suffer from the highest mortality rate in the world. One crucial factor contributing to the global disparity in mortality rate is long delay of diagnosis due to a severe shortage of trained pathologists, which consequently has led to a large proportion of late-stage presentation at diagnosis. To tackle this critical healthcare disparity, we have developed a deep learning-based diagnosis system for metastatic breast cancer that can achieve high diagnostic accuracy as well as computational efficiency and mobile readiness suitable for an under-resourced environment. We evaluated 4 Convolutional Neural Network (CNN) architectures: MobileNetV2, VGG16, ResNet50 and ResNet101. The MobileNetV2-based diagnostic model outperformed the more complex VGG16, ResNet50 and ResNet101 models in diagnostic accuracy, model generalization, and model training efficiency. The ROC AUC of MobilenetV2 (0.933, 95% CI: 0.930, 0.936) was higher than VGG16 (0.911, 95% CI: 0.908, 0.915), ResNet50 (0.869, 95% CI: 0.866, 0.873), and ResNet101 (0.873, 95% CI: 0.869, 0.876). The time per inference step for the MobileNetV2 model (15 ms/step) was substantially lower than that of VGG16 (48 ms/step), ResNet50 (37 ms/step), and ResNet110 (56 ms/step). The visual comparisons between the model prediction and ground truth have demonstrated that the MobileNetV2 diagnostic models can identify very small cancerous nodes embedded in a large area of normal cells which is challenging for manual image analysis. Equally Important, the light weight MobleNetV2 models were computationally efficient and ready for mobile devices or devices of low computational power. These advances empower the development of a resource-efficient and high performing AI-based metastatic breast cancer diagnostic system that can adapt to under-resourced healthcare facilities in developing countries.

Enhancing Fetal Anomaly Detection in Ultrasonography Images: A Review of Machine Learning-Based Approaches

Fetal development is a critical phase in prenatal care, demanding the timely identification of anomalies in ultrasound images to safeguard the well-being of both the unborn child and the mother. Medical imaging has played a pivotal role in detecting fetal abnormalities and malformations. However, despite significant advances in ultrasound technology, the accurate identification of irregularities in prenatal images continues to pose considerable challenges, often necessitating substantial time and expertise from medical professionals. In this review, we go through recent developments in machine learning (ML) methods applied to fetal ultrasound images. Specifically, we focus on a range of ML algorithms employed in the context of fetal ultrasound, encompassing tasks such as image classification, object recognition, and segmentation. We highlight how these innovative approaches can enhance ultrasound-based fetal anomaly detection and provide insights for future research and clinical implementations. Furthermore, we emphasize the need for further research in this domain where future investigations can contribute to more effective ultrasound-based fetal anomaly detection.

Deep learning-based high-accuracy detection for lumbar and cervical degenerative disease on T2-weighted MR images

PURPOSE

To develop and validate a deep learning (DL) model for detecting lumbar degenerative disease in both sagittal and axial views of T2-weighted MRI and evaluate its generalized performance in detecting cervical degenerative disease.

METHODS

T2-weighted MRI scans of 804 patients with symptoms of lumbar degenerative disease were retrospectively collected from three hospitals. The training dataset (n = 456) and internal validation dataset (n = 134) were randomly selected from the center I. Two external validation datasets comprising 100 and 114 patients were from center II and center III, respectively. A DL model based on 3D ResNet18 and transformer architecture was proposed to detect lumbar degenerative disease. In addition, a cervical MR image dataset comprising 200 patients from an independent hospital was used to evaluate the generalized performance of the DL model. The diagnostic performance was assessed by the free-response receiver operating characteristic (fROC) curve and precision-recall (PR) curve. Precision, recall, and F1-score were used to measure the DL model.

RESULTS

A total of 2497 three-dimension retrogression annotations were labeled for training (n = 1157) and multicenter validation (n = 1340). The DL model showed excellent detection efficiency in the internal validation dataset, with F1-score achieving 0.971 and 0.903 on the sagittal and axial MR images, respectively. Good performance was also observed in the external validation dataset I (F1-score, 0.768 on sagittal MR images and 0.837 on axial MR images) and external validation dataset II (F1-score, 0.787 on sagittal MR images and 0.770 on axial MR images). Furthermore, the robustness of the DL model was demonstrated via transfer learning and generalized performance evaluation on the external cervical dataset, with the F1-score yielding 0.931 and 0.919 on the sagittal and axial MR images, respectively.

CONCLUSION

The proposed DL model can automatically detect lumbar and cervical degenerative disease on T2-weighted MR images with good performance, robustness, and feasibility in clinical practice.

An Intelligent Attention-Based Transfer Learning Model for Accurate Differentiation of Bone Marrow Stains to Diagnose Hematological Disorder

Bone marrow (BM) is an essential part of the hematopoietic system, which generates all of the body's blood cells and maintains the body's overall health and immune system. The classification of bone marrow cells is pivotal in both clinical and research settings because many hematological diseases, such as leukemia, myelodysplastic syndromes, and anemias, are diagnosed based on specific abnormalities in the number, type, or morphology of bone marrow cells. There is a requirement for developing a robust deep-learning algorithm to diagnose bone marrow cells to keep a close check on them. This study proposes a framework for categorizing bone marrow cells into seven classes. In the proposed framework, five transfer learning models-DenseNet121, EfficientNetB5, ResNet50, Xception, and MobileNetV2-are implemented into the bone marrow dataset to classify them into seven classes. The best-performing DenseNet121 model was fine-tuned by adding one batch-normalization layer, one dropout layer, and two dense layers. The proposed fine-tuned DenseNet121 model was optimized using several optimizers, such as AdaGrad, AdaDelta, Adamax, RMSprop, and SGD, along with different batch sizes of 16, 32, 64, and 128. The fine-tuned DenseNet121 model was integrated with an attention mechanism to improve its performance by allowing the model to focus on the most relevant features or regions of the image, which can be particularly beneficial in medical imaging, where certain regions might have critical diagnostic information. The proposed fine-tuned and integrated DenseNet121 achieved the highest accuracy, with a training success rate of 99.97% and a testing success rate of 97.01%. The key hyperparameters, such as batch size, number of epochs, and different optimizers, were all considered for optimizing these pre-trained models to select the best model. This study will help in medical research to effectively classify the BM cells to prevent diseases like leukemia.

Flatfeet Severity-Level Detection Based on Alignment Measuring

Flat foot is a postural deformity in which the plantar part of the foot is either completely or partially contacted with the ground. In recent clinical practices, X-ray radiographs have been introduced to detect flat feet because they are more affordable to many clinics than using specialized devices. This research aims to develop an automated model that detects flat foot cases and their severity levels from lateral foot X-ray images by measuring three different foot angles: the Arch Angle, Meary's Angle, and the Calcaneal Inclination Angle. Since these angles are formed by connecting a set of points on the image, Template Matching is used to allocate a set of potential points for each angle, and then a classifier is used to select the points with the highest predicted likelihood to be the correct point. Inspired by literature, this research constructed and compared two models: a Convolutional Neural Network-based model and a Random Forest-based model. These models were trained on 8000 images and tested on 240 unseen cases. As a result, the highest overall accuracy rate was 93.13% achieved by the Random Forest model, with mean values for all foot types (normal foot, mild flat foot, and moderate flat foot) being: 93.38 precision, 92.56 recall, 96.46 specificity, 95.42 accuracy, and 92.90 F-Score. The main conclusions that were deduced from this research are: (1) Using transfer learning (VGG-16) as a feature-extractor-only, in addition to image augmentation, has greatly increased the overall accuracy rate. (2) Relying on three different foot angles shows more accurate estimations than measuring a single foot angle.

Deep Learning-Based Feature Extraction from Whole-Body PET/CT Employing Maximum Intensity Projection Images: Preliminary Results of Lung Cancer Data

Purpose

Deep learning (DL) has been widely used in various medical imaging analyses. Because of the difficulty in processing volume data, it is difficult to train a DL model as an end-to-end approach using PET volume as an input for various purposes including diagnostic classification. We suggest an approach employing two maximum intensity projection (MIP) images generated by whole-body FDG PET volume to employ pre-trained models based on 2-D images.

Methods

As a retrospective, proof-of-concept study, 562 [18F]FDG PET/CT images and clinicopathological factors of lung cancer patients were collected. MIP images of anterior and lateral views were used as inputs, and image features were extracted by a pre-trained convolutional neural network (CNN) model, ResNet-50. The relationship between the images was depicted on a parametric 2-D axes map using t-distributed stochastic neighborhood embedding (t-SNE), with clinicopathological factors.

Results

A DL-based feature map extracted by two MIP images was embedded by t-SNE. According to the visualization of the t-SNE map, PET images were clustered by clinicopathological features. The representative difference between the clusters of PET patterns according to the posture of a patient was visually identified. This map showed a pattern of clustering according to various clinicopathological factors including sex as well as tumor staging.

Conclusion

A 2-D image-based pre-trained model could extract image patterns of whole-body FDG PET volume by using anterior and lateral views of MIP images bypassing the direct use of 3-D PET volume that requires large datasets and resources. We suggest that this approach could be implemented as a backbone model for various applications for whole-body PET image analyses.

Development of a prediction model for head and neck volume reduction by clinical factors, dose-volume histogram parameters and radiomics in head and neck cancer†

In external radiotherapy of head and neck (HN) cancers, the reduction of irradiation accuracy due to HN volume reduction often causes a problem. Adaptive radiotherapy (ART) can effectively solve this problem; however, its application to all cases is impractical because of cost and time. Therefore, finding priority cases is essential. This study aimed to predict patients with HN cancers are more likely to need ART based on a quantitative measure of large HN volume reduction and evaluate model accuracy. The study included 172 cases of patients with HN cancer who received external irradiation. The HN volume was calculated using cone-beam computed tomography (CT) for irradiation-guided radiotherapy for all treatment fractions and classified into two groups: cases with a large reduction in the HN volume and cases without a large reduction. Radiomic features were extracted from the primary gross tumor volume (GTV) and nodal GTV of the planning CT. To develop the prediction model, four feature selection methods and two machine-learning algorithms were tested. Predictive performance was evaluated by the area under the curve (AUC), accuracy, sensitivity and specificity. Predictive performance was the highest for the random forest, with an AUC of 0.662. Furthermore, its accuracy, sensitivity and specificity were 0.692, 0.700 and 0.813, respectively. Selected features included radiomic features of the primary GTV, human papillomavirus in oropharyngeal cancer and the implementation of chemotherapy; thus, these features might be related to HN volume change. Our model suggested the potential to predict ART requirements based on HN volume reduction .

RANet: a custom CNN model and quanvolutional neural network for the automated detection of rheumatoid arthritis in hand thermal images

Rheumatoid arthritis is an autoimmune disease which affects the small joints. Early prediction of RA is necessary for the treatment and management of the disease. The current work presents a deep learning and quantum computing-based automated diagnostic approach for RA in hand thermal imaging. The study's goals are (i) to develop a custom RANet model and compare its performance with the pretrained models and quanvolutional neural network (QNN) to distinguish between the healthy subjects and RA patients, (ii) To validate the performance of the custom model using feature selection method and classification using machine learning (ML) classifiers. The present study developed a custom RANet model and employed pre-trained models such as ResNet101V2, InceptionResNetV2, and DenseNet201 to classify the RA patients and normal subjects. The deep features extracted from the RA Net model are fed into the ML classifiers after the feature selection process. The RANet model, RA Net+ SVM, and QNN model produced an accuracy of 95%, 97% and 93.33% respectively in the classification of healthy groups and RA patients. The developed RANet and QNN models based on thermal imaging could be employed as an accurate automated diagnostic tool to differentiate between the RA and control groups.

Hybrid morphological-convolutional neural networks for computer-aided diagnosis

Training deep Convolutional Neural Networks (CNNs) presents challenges in terms of memory requirements and computational resources, often resulting in issues such as model overfitting and lack of generalization. These challenges can only be mitigated by using an excessive number of training images. However, medical image datasets commonly suffer from data scarcity due to the complexities involved in their acquisition, preparation, and curation. To address this issue, we propose a compact and hybrid machine learning architecture based on the Morphological and Convolutional Neural Network (MCNN), followed by a Random Forest classifier. Unlike deep CNN architectures, the MCNN was specifically designed to achieve effective performance with medical image datasets limited to a few hundred samples. It incorporates various morphological operations into a single layer and uses independent neural networks to extract information from each signal channel. The final classification is obtained by utilizing a Random Forest classifier on the outputs of the last neural network layer. We compare the classification performance of our proposed method with three popular deep CNN architectures (ResNet-18, ShuffleNet-V2, and MobileNet-V2) using two training approaches: full training and transfer learning. The evaluation was conducted on two distinct medical image datasets: the ISIC dataset for melanoma classification and the ORIGA dataset for glaucoma classification. Results demonstrate that the MCNN method exhibits reliable performance in melanoma classification, achieving an AUC of 0.94 (95% CI: 0.91 to 0.97), outperforming the popular CNN architectures. For the glaucoma dataset, the MCNN achieved an AUC of 0.65 (95% CI: 0.53 to 0.74), which was similar to the performance of the popular CNN architectures. This study contributes to the understanding of mathematical morphology in shallow neural networks for medical image classification and highlights the potential of hybrid architectures in effectively learning from medical image datasets that are limited by a small number of case samples.

Expert-level pediatric brain tumor segmentation in a limited data scenario with stepwise transfer learning

Purpose

Artificial intelligence (AI)-automated tumor delineation for pediatric gliomas would enable real-time volumetric evaluation to support diagnosis, treatment response assessment, and clinical decision-making. Auto-segmentation algorithms for pediatric tumors are rare, due to limited data availability, and algorithms have yet to demonstrate clinical translation.

Methods

We leveraged two datasets from a national brain tumor consortium (n=184) and a pediatric cancer center (n=100) to develop, externally validate, and clinically benchmark deep learning neural networks for pediatric low-grade glioma (pLGG) segmentation using a novel in-domain, stepwise transfer learning approach. The best model [via Dice similarity coefficient (DSC)] was externally validated and subject to randomized, blinded evaluation by three expert clinicians wherein clinicians assessed clinical acceptability of expert- and AI-generated segmentations via 10-point Likert scales and Turing tests.

Results

The best AI model utilized in-domain, stepwise transfer learning (median DSC: 0.877 [IQR 0.715-0.914]) versus baseline model (median DSC 0.812 [IQR 0.559-0.888]; p<0.05). On external testing (n=60), the AI model yielded accuracy comparable to inter-expert agreement (median DSC: 0.834 [IQR 0.726-0.901] vs. 0.861 [IQR 0.795-0.905], p=0.13). On clinical benchmarking (n=100 scans, 300 segmentations from 3 experts), the experts rated the AI model higher on average compared to other experts (median Likert rating: 9 [IQR 7-9]) vs. 7 [IQR 7-9], p<0.05 for each). Additionally, the AI segmentations had significantly higher (p<0.05) overall acceptability compared to experts on average (80.2% vs. 65.4%). Experts correctly predicted the origins of AI segmentations in an average of 26.0% of cases.

Conclusions

Stepwise transfer learning enabled expert-level, automated pediatric brain tumor auto-segmentation and volumetric measurement with a high level of clinical acceptability. This approach may enable development and translation of AI imaging segmentation algorithms in limited data scenarios.

Machine learning in the identification of prognostic DNA methylation biomarkers among patients with cancer: A systematic review of epigenome-wide studies

BACKGROUND

DNA methylation biomarkers have great potential in improving prognostic classification systems for patients with cancer. Machine learning (ML)-based analytic techniques might help overcome the challenges of analyzing high-dimensional data in relatively small sample sizes. This systematic review summarizes the current use of ML-based methods in epigenome-wide studies for the identification of DNA methylation signatures associated with cancer prognosis.

METHODS

We searched three electronic databases including PubMed, EMBASE, and Web of Science for articles published until 2 January 2023. ML-based methods and workflows used to identify DNA methylation signatures associated with cancer prognosis were extracted and summarized. Two authors independently assessed the methodological quality of included studies by a seven-item checklist adapted from 'A Tool to Assess Risk of Bias and Applicability of Prediction Model Studies (PROBAST)' and from the 'Reporting Recommendations for Tumor Marker Prognostic Studies (REMARK). Different ML methods and workflows used in included studies were summarized and visualized by a sunburst chart, a bubble chart, and Sankey diagrams, respectively.

RESULTS

Eighty-three studies were included in this review. Three major types of ML-based workflows were identified. 1) unsupervised clustering, 2) supervised feature selection, and 3) deep learning-based feature transformation. For the three workflows, the most frequently used ML techniques were consensus clustering, least absolute shrinkage and selection operator (LASSO), and autoencoder, respectively. The systematic review revealed that the performance of these approaches has not been adequately evaluated yet and that methodological and reporting flaws were common in the identified studies using ML techniques.

CONCLUSIONS

There is great heterogeneity in ML-based methodological strategies used by epigenome-wide studies to identify DNA methylation markers associated with cancer prognosis. In theory, most existing workflows could not handle the high multi-collinearity and potentially non-linearity interactions in epigenome-wide DNA methylation data. Benchmarking studies are needed to compare the relative performance of various approaches for specific cancer types. Adherence to relevant methodological and reporting guidelines are urgently needed.

Clustering-based spatial analysis (CluSA) framework through graph neural network for chronic kidney disease prediction using histopathology images

Machine learning applied to digital pathology has been increasingly used to assess kidney function and diagnose the underlying cause of chronic kidney disease (CKD). We developed a novel computational framework, clustering-based spatial analysis (CluSA), that leverages unsupervised learning to learn spatial relationships between local visual patterns in kidney tissue. This framework minimizes the need for time-consuming and impractical expert annotations. 107,471 histopathology images obtained from 172 biopsy cores were used in the clustering and in the deep learning model. To incorporate spatial information over the clustered image patterns on the biopsy sample, we spatially encoded clustered patterns with colors and performed spatial analysis through graph neural network. A random forest classifier with various groups of features were used to predict CKD. For predicting eGFR at the biopsy, we achieved a sensitivity of 0.97, specificity of 0.90, and accuracy of 0.95. AUC was 0.96. For predicting eGFR changes in one-year, we achieved a sensitivity of 0.83, specificity of 0.85, and accuracy of 0.84. AUC was 0.85. This study presents the first spatial analysis based on unsupervised machine learning algorithms. Without expert annotation, CluSA framework can not only accurately classify and predict the degree of kidney function at the biopsy and in one year, but also identify novel predictors of kidney function and renal prognosis.

Multiparametric MRI: From Simultaneous Rapid Acquisition Methods and Analysis Techniques Using Scoring, Machine Learning, Radiomics, and Deep Learning to the Generation of Novel Metrics

ABSTRACT

With the recent advancements in rapid imaging methods, higher numbers of contrasts and quantitative parameters can be acquired in less and less time. Some acquisition models simultaneously obtain multiparametric images and quantitative maps to reduce scan times and avoid potential issues associated with the registration of different images. Multiparametric magnetic resonance imaging (MRI) has the potential to provide complementary information on a target lesion and thus overcome the limitations of individual techniques. In this review, we introduce methods to acquire multiparametric MRI data in a clinically feasible scan time with a particular focus on simultaneous acquisition techniques, and we discuss how multiparametric MRI data can be analyzed as a whole rather than each parameter separately. Such data analysis approaches include clinical scoring systems, machine learning, radiomics, and deep learning. Other techniques combine multiple images to create new quantitative maps associated with meaningful aspects of human biology. They include the magnetic resonance g-ratio, the inner to the outer diameter of a nerve fiber, and the aerobic glycolytic index, which captures the metabolic status of tumor tissues.

Automated T1 and T2 mapping segmentation on cardiovascular magnetic resonance imaging using deep learning

Introduction

Structural and functional heart abnormalities can be examined non-invasively with cardiac magnetic resonance imaging (CMR). Thanks to the development of MR devices, diagnostic scans can capture more and more relevant information about possible heart diseases. T1 and T2 mapping are such novel technology, providing tissue specific information even without the administration of contrast material. Artificial intelligence solutions based on deep learning have demonstrated state-of-the-art results in many application areas, including medical imaging. More specifically, automated tools applied at cine sequences have revolutionized volumetric CMR reporting in the past five years. Applying deep learning models to T1 and T2 mapping images can similarly improve the efficiency of post-processing pipelines and consequently facilitate diagnostic processes.

Methods

In this paper, we introduce a deep learning model for myocardium segmentation trained on over 7,000 raw CMR images from 262 subjects of heterogeneous disease etiology. The data were labeled by three experts. As part of the evaluation, Dice score and Hausdorff distance among experts is calculated, and the expert consensus is compared with the model's predictions.

Results

Our deep learning method achieves 86% mean Dice score, while contours provided by three experts on the same data show 90% mean Dice score. The method's accuracy is consistent across epicardial and endocardial contours, and on basal, midventricular slices, with only 5% lower results on apical slices, which are often challenging even for experts.

Conclusions

We trained and evaluated a deep learning based segmentation model on 262 heterogeneous CMR cases. Applying deep neural networks to T1 and T2 mapping could similarly improve diagnostic practices. Using the fine details of T1 and T2 mapping images and high-quality labels, the objective of this research is to approach human segmentation accuracy with deep learning.

Revisiting Transfer Learning Method for Tuberculosis Diagnosis

Transfer learning (TL) has been proven to be a good strategy for solving domain-specific problems in many deep learning (DL) applications. Typically, in TL, a pre-trained DL model is used as a feature extractor and the extracted features are then fed to a newly trained classifier as the model head. In this study, we propose a new ensemble approach of transfer learning that uses multiple neural network classifiers at once in the model head. We compared the classification results of the proposed ensemble approach with the direct approach of several popular models, namely VGG-16, ResNet-50, and MobileNet, on two publicly available tuberculosis datasets, i.e., Montgomery County (MC) and Shenzhen (SZ) datasets. Moreover, we also compared the results when a fully pre-trained DL model was used for feature extraction versus the cases in which the features were obtained from a middle layer of the pre-trained DL model. Several metrics derived from confusion matrix results were used, namely the accuracy (ACC), sensitivity (SNS), specificity (SPC), precision (PRC), and F1-score. We concluded that the proposed ensemble approach outperformed the direct approach. Best result was achieved by ResNet-50 when the features were extracted from a middle layer with an accuracy of 91.2698% on MC dataset.Clinical Relevance- The proposed ensemble approach could increase the detection accuracy of 7-8% for Montgomery County dataset and 4-5% for Shenzhen dataset.