Transfer learning for medical image classification: a literature review

Data-driven prediction of spinal cord injury recovery: An exploration of current status and future perspectives

Spinal Cord Injury (SCI) presents a significant challenge in rehabilitation medicine, with recovery outcomes varying widely among individuals. Machine learning (ML) is a promising approach to enhance the prediction of recovery trajectories, but its integration into clinical practice requires a thorough understanding of its efficacy and applicability. We systematically reviewed the current literature on data-driven models of SCI recovery prediction. The included studies were evaluated based on a range of criteria assessing the approach, implementation, input data preferences, and the clinical outcomes aimed to forecast. We observe a tendency to utilize routinely acquired data, such as International Standards for Neurological Classification of SCI (ISNCSCI), imaging, and demographics, for the prediction of functional outcomes derived from the Spinal Cord Independence Measure (SCIM) III and Functional Independence Measure (FIM) scores with a focus on motor ability. Although there has been an increasing interest in data-driven studies over time, traditional machine learning architectures, such as linear regression and tree-based approaches, remained the overwhelmingly popular choices for implementation. This implies ample opportunities for exploring architectures addressing the challenges of predicting SCI recovery, including techniques for learning from limited longitudinal data, improving generalizability, and enhancing reproducibility. We conclude with a perspective, highlighting possible future directions for data-driven SCI recovery prediction and drawing parallels to other application fields in terms of diverse data types (imaging, tabular, sequential, multimodal), data challenges (limited, missing, longitudinal data), and algorithmic needs (causal inference, robustness).

Using the GoogLeNet deep-learning model to distinguish between benign and malignant breast masses based on conventional ultrasound: a systematic review and meta-analysis

Background

Breast cancer is one of the most common malignancies in women worldwide, and early and accurate diagnosis is crucial for improving treatment outcomes. Conventional ultrasound (CUS) is a widely used screening method for breast cancer; however, the subjective nature of interpreting the results can lead to diagnostic errors. The current study sought to estimate the effectiveness of using a GoogLeNet deep-learning convolutional neural network (CNN) model to identify benign and malignant breast masses based on CUS.

Methods

A literature search was conducted of the Embase, PubMed, Web of Science, Wanfang, China National Knowledge Infrastructure (CNKI), and other databases to retrieve studies related to GoogLeNet deep-learning CUS-based models published before July 15, 2023. The diagnostic performance of the GoogLeNet models was evaluated using several metrics, including pooled sensitivity (PSEN), pooled specificity (PSPE), the positive likelihood ratio (PLR), the negative likelihood ratio (NLR), the diagnostic odds ratio (DOR), and the area under the curve (AUC). The quality of the included studies was evaluated using the Quality Assessment of Diagnostic Accuracy Studies Scale (QUADAS). The eligibility of the included literature were independently searched and assessed by two authors.

Results

All of the 12 studies that used pathological findings as the gold standard were included in the meta-analysis. The overall average estimation of sensitivity and specificity was 0.85 [95% confidence interval (CI): 0.80-0.89] and 0.86 (95% CI: 0.78-0.92), respectively. The PLR and NLR were 6.2 (95% CI: 3.9-9.9) and 0.17 (95% CI: 0.12-0.23), respectively. The DOR was 37.06 (95% CI: 20.78-66.10). The AUC was 0.92 (95% CI: 0.89-0.94). No obvious publication bias was detected.

Conclusions

The GoogLeNet deep-learning model, which uses a CNN, achieved good diagnostic results in distinguishing between benign and malignant breast masses in CUS-based images.

Applications of machine learning to MR imaging of pediatric low-grade gliomas

INTRODUCTION

Machine learning (ML) shows promise for the automation of routine tasks related to the treatment of pediatric low-grade gliomas (pLGG) such as tumor grading, typing, and segmentation. Moreover, it has been shown that ML can identify crucial information from medical images that is otherwise currently unattainable. For example, ML appears to be capable of preoperatively identifying the underlying genetic status of pLGG.

METHODS

In this chapter, we reviewed, to the best of our knowledge, all published works that have used ML techniques for the imaging-based evaluation of pLGGs. Additionally, we aimed to provide some context on what it will take to go from the exploratory studies we reviewed to clinically deployed models.

RESULTS

Multiple studies have demonstrated that ML can accurately grade, type, and segment and detect the genetic status of pLGGs. We compared the approaches used between the different studies and observed a high degree of variability throughout the methodologies. Standardization and cooperation between the numerous groups working on these approaches will be key to accelerating the clinical deployment of these models.

CONCLUSION

The studies reviewed in this chapter detail the potential for ML techniques to transform the treatment of pLGG. However, there are still challenges that need to be overcome prior to clinical deployment.

Enhancing pap smear image classification: integrating transfer learning and attention mechanisms for improved detection of cervical abnormalities

Cervical cancer remains a major global health challenge, accounting for significant morbidity and mortality among women. Early detection through screening, such as Pap smear tests, is crucial for effective treatment and improved patient outcomes. However, traditional manual analysis of Pap smear images is labor-intensive, subject to human error, and requires extensive expertise. To address these challenges, automated approaches using deep learning techniques have been increasingly explored, offering the potential for enhanced diagnostic accuracy and efficiency. This research focuses on improving cervical cancer detection from Pap smear images using advanced deep-learning techniques. Specifically, we aim to enhance classification performance by leveraging Transfer Learning (TL) combined with an attention mechanism, supplemented by effective preprocessing techniques. Our preprocessing pipeline includes image normalization, resizing, and the application of Histogram of Oriented Gradients (HOG), all of which contribute to better feature extraction and improved model performance. The dataset used in this study is the Mendeley Liquid-Based Cytology (LBC) dataset, which provides a comprehensive collection of cervical cytology images annotated by expert cytopathologists. Initial experiments with the ResNet model on raw data yielded an accuracy of 63.95%. However, by applying our preprocessing techniques and integrating an attention mechanism, the accuracy of the ResNet model increased dramatically to 96.74%. Further, the Xception model, known for its superior feature extraction capabilities, achieved the best performance with an accuracy of 98.95%, along with high precision (0.97), recall (0.99), and F1-Score (0.98) on preprocessed data with an attention mechanism. These results underscore the effectiveness of combining preprocessing techniques, TL, and attention mechanisms to significantly enhance the performance of automated cervical cancer detection systems. Our findings demonstrate the potential of these advanced techniques to provide reliable, accurate, and efficient diagnostic tools, which could greatly benefit clinical practice and improve patient outcomes in cervical cancer screening.

3-1-3 Weight averaging technique-based performance evaluation of deep neural networks for Alzheimer's disease detection using structural MRI

Alzheimer's disease (AD) is a progressive neurological disorder. It is identified by the gradual shrinkage of the brain and the loss of brain cells. This leads to cognitive decline and impaired social functioning, making it a major contributor to dementia. While there are no treatments to reverse AD's progression, spotting the disease's onset can have a significant impact in the medical field. Deep learning (DL) has revolutionized medical image classification by automating feature engineering, removing the requirement for human experts in feature extraction. DL-based solutions are highly accurate but demand a lot of training data, which poses a common challenge. Transfer learning (TL) has gained attention for its knack for handling limited data and expediting model training. This study uses TL to classify AD using T1-weighted 3D Magnetic Resonance Imaging (MRI) from the Alzheimer's Disease Neuroimaging (ADNI) database. Four modified pre-trained deep neural networks (DNN), VGG16, MobileNet, DenseNet121, and NASNetMobile, are trained and evaluated on the ADNI dataset. The 3-1-3 weight averaging technique and fine-tuning improve the performance of the classification models. The evaluated accuracies for AD classification are VGG16: 98.75%; MobileNet: 97.5%; DenseNet: 97.5%; and NASNetMobile: 96.25%. The receiver operating characteristic (ROC), precision-recall (PR), and Kolmogorov-Smirnov (KS) statistic plots validate the effectiveness of the modified pre-trained model. Modified VGG16 excels with area under the curve (AUC) values of 0.99 for ROC and 0.998 for PR curves. The proposed approach shows effective AD classification by achieving high accuracy using the 3-1-3 weight averaging technique and fine-tuning.

Applications of Artificial Intelligence and Machine Learning in Spine MRI

Diagnostic imaging, particularly MRI, plays a key role in the evaluation of many spine pathologies. Recent progress in artificial intelligence and its subset, machine learning, has led to many applications within spine MRI, which we sought to examine in this review. A literature search of the major databases (PubMed, MEDLINE, Web of Science, ClinicalTrials.gov) was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. The search yielded 1226 results, of which 50 studies were selected for inclusion. Key data from these studies were extracted. Studies were categorized thematically into the following: Image Acquisition and Processing, Segmentation, Diagnosis and Treatment Planning, and Patient Selection and Prognostication. Gaps in the literature and the proposed areas of future research are discussed. Current research demonstrates the ability of artificial intelligence to improve various aspects of this field, from image acquisition to analysis and clinical care. We also acknowledge the limitations of current technology. Future work will require collaborative efforts in order to fully exploit new technologies while addressing the practical challenges of generalizability and implementation. In particular, the use of foundation models and large-language models in spine MRI is a promising area, warranting further research. Studies assessing model performance in real-world clinical settings will also help uncover unintended consequences and maximize the benefits for patient care.

Preoperative prediction of renal fibrous capsule invasion in clear cell renal cell carcinoma using CT-based radiomics model

OBJECTIVES

To develop radiomics-based classifiers for preoperative prediction of fibrous capsule invasion in renal cell carcinoma (RCC) patients by CT images.

METHODS

In this study, clear cell RCC (ccRCC) patients who underwent both preoperative abdominal contrast-enhanced CT and nephrectomy surgery at our hospital were analysed. By transfer learning, we used base model obtained from Kidney Tumour Segmentation challenge dataset to semi-automatically segment kidney and tumours from corticomedullary phase (CMP) CT images. Dice similarity coefficient (DSC) was measured to evaluate the performance of segmentation models. Ten machine learning classifiers were compared in our study. Performance of the models was assessed by their accuracy, precision, recall, and area under the receiver operating characteristic curve (AUC). The reporting and methodological quality of our study was assessed by the CLEAR checklist and METRICS score.

RESULTS

This retrospective study enrolled 163 ccRCC patients. The semiautomatic segmentation model using CMP CT images obtained DSCs of 0.98 in the training cohort and 0.96 in the test cohort for kidney segmentation, and DSCs of 0.94 and 0.86 for tumour segmentation in the training and test set, respectively. For preoperative prediction of renal capsule invasion, the AdaBoost had the best performance in batch 1, with accuracy, precision, recall, and F1-score equal to 0.8571, 0.8333, 0.9091, and 0.8696, respectively; and the same classifier was also the most suitable for this classification in batch 2. The AUCs of AdaBoost for batch 1 and batch 2 were 0.83 (95% CI: 0.68-0.98) and 0.74 (95% CI: 0.51-0.97), respectively. Nine common significant features for classification were found from 2 independent batch datasets, including morphological and texture features.

CONCLUSIONS

The CT-based radiomics classifiers performed well for the preoperative prediction of fibrous capsule invasion in ccRCC.

ADVANCES IN KNOWLEDGE

Noninvasive prediction of renal fibrous capsule invasion in RCC is rather difficult by abdominal CT images before surgery. A machine learning classifier integrated with radiomics features shows a promising potential to assist surgical treatment options for RCC patients.

Deep learning with uncertainty estimation for automatic tumor segmentation in PET/CT of head and neck cancers: impact of model complexity, image processing and augmentation

Objective.Target volumes for radiotherapy are usually contoured manually, which can be time-consuming and prone to inter- and intra-observer variability. Automatic contouring by convolutional neural networks (CNN) can be fast and consistent but may produce unrealistic contours or miss relevant structures. We evaluate approaches for increasing the quality and assessing the uncertainty of CNN-generated contours of head and neck cancers with PET/CT as input.Approach.Two patient cohorts with head and neck squamous cell carcinoma and baseline18F-fluorodeoxyglucose positron emission tomography and computed tomography images (FDG-PET/CT) were collected retrospectively from two centers. The union of manual contours of the gross primary tumor and involved nodes was used to train CNN models for generating automatic contours. The impact of image preprocessing, image augmentation, transfer learning and CNN complexity, architecture, and dimension (2D or 3D) on model performance and generalizability across centers was evaluated. A Monte Carlo dropout technique was used to quantify and visualize the uncertainty of the automatic contours.Main results. CNN models provided contours with good overlap with the manually contoured ground truth (median Dice Similarity Coefficient: 0.75-0.77), consistent with reported inter-observer variations and previous auto-contouring studies. Image augmentation and model dimension, rather than model complexity, architecture, or advanced image preprocessing, had the largest impact on model performance and cross-center generalizability. Transfer learning on a limited number of patients from a separate center increased model generalizability without decreasing model performance on the original training cohort. High model uncertainty was associated with false positive and false negative voxels as well as low Dice coefficients.Significance.High quality automatic contours can be obtained using deep learning architectures that are not overly complex. Uncertainty estimation of the predicted contours shows potential for highlighting regions of the contour requiring manual revision or flagging segmentations requiring manual inspection and intervention.

AI Use in Mammography for Diagnosing Metachronous Contralateral Breast Cancer

Although several studies have been conducted on artificial intelligence (AI) use in mammography (MG), there is still a paucity of research on the diagnosis of metachronous bilateral breast cancer (BC), which is typically more challenging to diagnose. This study aimed to determine whether AI could enhance BC detection, achieving earlier or more accurate diagnoses than radiologists in cases of metachronous contralateral BC. We included patients who underwent unilateral BC surgery and subsequently developed contralateral BC. This retrospective study evaluated the AI-supported MG diagnostic system called FxMammo™. We evaluated the capability of FxMammo™ (FathomX Pte Ltd., Singapore) to diagnose BC more accurately or earlier than radiologists' assessments. This evaluation was supplemented by reviewing MG readings made by radiologists. Out of 1101 patients who underwent surgery, 10 who had initially undergone a partial mastectomy and later developed contralateral BC were analyzed. The AI system identified malignancies in six cases (60%), while radiologists identified five cases (50%). Notably, two cases (20%) were diagnosed solely by the AI system. Additionally, for these cases, the AI system had identified malignancies a year before the conventional diagnosis. This study highlights the AI system's effectiveness in diagnosing metachronous contralateral BC via MG. In some cases, the AI system consistently diagnosed cancer earlier than radiological assessments.

Dense Convolutional Neural Network-Based Deep Learning Pipeline for Pre-Identification of Circular Leaf Spot Disease of Diospyros kaki Leaves Using Optical Coherence Tomography

Circular leaf spot (CLS) disease poses a significant threat to persimmon cultivation, leading to substantial harvest reductions. Existing visual and destructive inspection methods suffer from subjectivity, limited accuracy, and considerable time consumption. This study presents an automated pre-identification method of the disease through a deep learning (DL) based pipeline integrated with optical coherence tomography (OCT), thereby addressing the highlighted issues with the existing methods. The investigation yielded promising outcomes by employing transfer learning with pre-trained DL models, specifically DenseNet-121 and VGG-16. The DenseNet-121 model excels in differentiating among three stages of CLS disease (healthy (H), apparently healthy (or healthy-infected (HI)), and infected (I)). The model achieved precision values of 0.7823 for class-H, 0.9005 for class-HI, and 0.7027 for class-I, supported by recall values of 0.8953 for class-HI and 0.8387 for class-I. Moreover, the performance of CLS detection was enhanced by a supplemental quality inspection model utilizing VGG-16, which attained an accuracy of 98.99% in discriminating between low-detail and high-detail images. Moreover, this study employed a combination of LAMP and A-scan for the dataset labeling process, significantly enhancing the accuracy of the models. Overall, this study underscores the potential of DL techniques integrated with OCT to enhance disease identification processes in agricultural settings, particularly in persimmon cultivation, by offering efficient and objective pre-identification of CLS and enabling early intervention and management strategies.

Harnessing Deep Learning for Accurate Pathological Assessment of Brain Tumor Cell Types

Primary diffuse central nervous system large B-cell lymphoma (CNS-pDLBCL) and high-grade glioma (HGG) often present similarly, clinically and on imaging, making differentiation challenging. This similarity can complicate pathologists' diagnostic efforts, yet accurately distinguishing between these conditions is crucial for guiding treatment decisions. This study leverages a deep learning model to classify brain tumor pathology images, addressing the common issue of limited medical imaging data. Instead of training a convolutional neural network (CNN) from scratch, we employ a pre-trained network for extracting deep features, which are then used by a support vector machine (SVM) for classification. Our evaluation shows that the Resnet50 (TL + SVM) model achieves a 97.4% accuracy, based on tenfold cross-validation on the test set. These results highlight the synergy between deep learning and traditional diagnostics, potentially setting a new standard for accuracy and efficiency in the pathological diagnosis of brain tumors.

Dual-channel end-to-end network with prior knowledge embedding for improving spatial resolution of magnetic particle imaging

Magnetic particle imaging (MPI) is an emerging non-invasive medical imaging tomography technology based on magnetic particles, with excellent imaging depth penetration, high sensitivity and contrast. Spatial resolution and signal-to-noise ratio (SNR) are key performance metrics for evaluating MPI, which are directly influenced by the gradient of the selection field (SF). Increasing the SF gradient can improve the spatial resolution of MPI, but will lead to a decrease in SNR. Deep learning (DL) methods may enable obtaining high-resolution images from low-resolution images to improve the MPI resolution under low gradient conditions. However, existing DL methods overlook the physical procedures contributing to the blurring of MPI images, resulting in low interpretability and hindering breakthroughs in resolution. To address this issue, we propose a dual-channel end-to-end network with prior knowledge embedding for MPI (DENPK-MPI) to effectively establish a latent mapping between low-gradient and high-gradient images, thus improving MPI resolution without compromising SNR. By seamlessly integrating MPI PSF with DL paradigm, DENPK-MPI leads to a significant improvement in spatial resolution performance. Simulation, phantom, and in vivo MPI experiments have collectively confirmed that our method can improve the resolution of low-gradient MPI images without sacrificing SNR, resulting in a decrease in full width at half maximum by 14.8%-23.8 %, and the accuracy of image reconstruction is 18.2 %-27.3 % higher than other DL methods. In conclusion, we propose a DL method that incorporates MPI prior knowledge, which can improve the spatial resolution of MPI without compromising SNR and possess improved biomedical application.

Comparative study of DCNN and image processing based classification of chest X-rays for identification of COVID-19 patients using fine-tuning

The conventional detection of COVID-19 by evaluating the CT scan images is tiresome, often experiences high inter-observer variability and uncertainty issues. This work proposes the automatic detection and classification of COVID-19 by analysing the chest X-ray images (CXR) with the deep convolutional neural network (DCNN) models through a fine-tuning and pre-training approach. CXR images pertaining to four health scenarios, namely, healthy, COVID-19, bacterial pneumonia and viral pneumonia, are considered and subjected to data augmentation. Two types of input datasets are prepared; in which dataset I contains the original image dataset categorised under four classes, whereas the original CXR images are subjected to image pre-processing via Contrast Limited Adaptive Histogram Equalisation (CLAHE) algorithm and Blackhat Morphological Operation (BMO) for devising the input dataset II. Both datasets are supplied as input to various DCNN models such as DenseNet, MobileNet, ResNet, VGG16, and Xception for achieving multi-class classification. It is observed that the classification accuracies are improved, and the classification errors are reduced with the image pre-processing. Overall, the VGG16 model resulted in better classification accuracies and reduced classification errors while accomplishing multi-class classification. Thus, the proposed work would assist the clinical diagnosis, and reduce the workload of the front-line healthcare workforce and medical professionals.

Assessing the Influence of B-US, CDFI, SE, and Patient Age on Predicting Molecular Subtypes in Breast Lesions Using Deep Learning Algorithms

OBJECTIVES

Our study aims to investigate the impact of B-mode ultrasound (B-US) imaging, color Doppler flow imaging (CDFI), strain elastography (SE), and patient age on the prediction of molecular subtypes in breast lesions.

METHODS

Totally 2272 multimodal ultrasound imaging was collected from 198 patients. The ResNet-18 network was employed to predict four molecular subtypes from B-US imaging, CDFI, and SE of patients with different ages. All the images were split into training and testing datasets by the ratio of 80%:20%. The predictive performance on testing dataset was evaluated through 5 metrics including mean accuracy, precision, recall, F1-scores, and confusion matrix.

RESULTS

Based on B-US imaging, the test mean accuracy is 74.50%, the precision is 74.84%, the recall is 72.48%, and the F1-scores is 0.73. By combining B-US imaging with CDFI, the results were increased to 85.41%, 85.03%, 85.05%, and 0.84, respectively. With the integration of B-US imaging and SE, the results were changed to 75.64%, 74.69%, 73.86%, and 0.74, respectively. Using images from patients under 40 years old, the results were 90.48%, 90.88%, 88.47%, and 0.89. When images from patients who are above 40 years old, they were changed to 81.96%, 83.12%, 80.5%, and 0.81, respectively.

CONCLUSION

Multimodal ultrasound imaging can be used to accurately predict the molecular subtypes of breast lesions. In addition to B-US imaging, CDFI rather than SE contribute further to improve predictive performance. The predictive performance is notably better for patients under 40 years old compared with those who are 40 years old and above.

A deep learning approach to direct immunofluorescence pattern recognition in autoimmune bullous diseases

BACKGROUND

Artificial intelligence (AI) is reshaping healthcare, using machine and deep learning (DL) to enhance disease management. Dermatology has seen improved diagnostics, particularly in skin cancer detection, through the integration of AI. However, the potential of AI in automating immunofluorescence imaging for autoimmune bullous skin diseases (AIBDs) remains untapped. While direct immunofluorescence (DIF) supports diagnosis, its manual interpretation can hinder efficiency. The use of DL to classify DIF patterns automatically, including the intercellular (ICP) and linear pattern (LP), holds promise for improving the diagnosis of AIBDs.

OBJECTIVES

To develop AI algorithms for automated classification of AIBD DIF patterns, such as ICP and LP, in order to enhance diagnostic accuracy, streamline disease management and improve patient outcomes through DL-driven immunofluorescence interpretation.

METHODS

We collected immunofluorescence images from skin biopsies of patients suspected of having an AIBD between January 2022 and January 2024. Skin tissue was obtained via a 5-mm punch biopsy, prepared for DIF. Experienced dermatologists classified the images as ICP, LP or negative. To evaluate our DL approach, we divided the images into training (n = 436) and test sets (n = 93). We employed transfer learning with pretrained deep neural networks and conducted fivefold cross-validation to assess model performance. Our dataset's class imbalance was addressed using weighted loss and data augmentation strategies. The models were trained for 50 epochs using Pytorch, achieving an image size of 224 × 224 pixels for both convolutional neural networks (CNNs) and the Swin Transformer.

RESULTS

Our study compared six CNNs and the Swin Transformer for AIBD image classification, with the Swin Transformer achieving the highest average validation accuracy (98.5%). On a separate test set, the best model attained an accuracy of 94.6%, demonstrating 95.3% sensitivity and 97.5% specificity across AIBD classes. Visualization with Grad-CAM (class activation mapping) highlighted the model's reliance on characteristic patterns for accurate classification.

CONCLUSIONS

The study highlighted the accuracy of CNNs in identifying DIF features. This approach aids automated analysis and reporting, offering reproducibility, speed, data handling and cost-efficiency. Integrating DL into skin immunofluorescence promises precise diagnostics and streamlined reporting in this branch of dermatology.

Selection of pre-trained weights for transfer learning in automated cytomegalovirus retinitis classification

Cytomegalovirus retinitis (CMVR) is a significant cause of vision loss. Regular screening is crucial but challenging in resource-limited settings. A convolutional neural network is a state-of-the-art deep learning technique to generate automatic diagnoses from retinal images. However, there are limited numbers of CMVR images to train the model properly. Transfer learning (TL) is a strategy to train a model with a scarce dataset. This study explores the efficacy of TL with different pre-trained weights for automated CMVR classification using retinal images. We utilised a dataset of 955 retinal images (524 CMVR and 431 normal) from Siriraj Hospital, Mahidol University, collected between 2005 and 2015. Images were processed using Kowa VX-10i or VX-20 fundus cameras and augmented for training. We employed DenseNet121 as a backbone model, comparing the performance of TL with weights pre-trained on ImageNet, APTOS2019, and CheXNet datasets. The models were evaluated based on accuracy, loss, and other performance metrics, with the depth of fine-tuning varied across different pre-trained weights. The study found that TL significantly enhances model performance in CMVR classification. The best results were achieved with weights sequentially transferred from ImageNet to APTOS2019 dataset before application to our CMVR dataset. This approach yielded the highest mean accuracy (0.99) and lowest mean loss (0.04), outperforming other methods. The class activation heatmaps provided insights into the model's decision-making process. The model with APTOS2019 pre-trained weights offered the best explanation and highlighted the pathologic lesions resembling human interpretation. Our findings demonstrate the potential of sequential TL in improving the accuracy and efficiency of CMVR diagnosis, particularly in settings with limited data availability. They highlight the importance of domain-specific pre-training in medical image classification. This approach streamlines the diagnostic process and paves the way for broader applications in automated medical image analysis, offering a scalable solution for early disease detection.

Utilizing Deep Feature Fusion for Automatic Leukemia Classification: An Internet of Medical Things-Enabled Deep Learning Framework

Acute lymphoblastic leukemia, commonly referred to as ALL, is a type of cancer that can affect both the blood and the bone marrow. The process of diagnosis is a difficult one since it often calls for specialist testing, such as blood tests, bone marrow aspiration, and biopsy, all of which are highly time-consuming and expensive. It is essential to obtain an early diagnosis of ALL in order to start therapy in a timely and suitable manner. In recent medical diagnostics, substantial progress has been achieved through the integration of artificial intelligence (AI) and Internet of Things (IoT) devices. Our proposal introduces a new AI-based Internet of Medical Things (IoMT) framework designed to automatically identify leukemia from peripheral blood smear (PBS) images. In this study, we present a novel deep learning-based fusion model to detect ALL types of leukemia. The system seamlessly delivers the diagnostic reports to the centralized database, inclusive of patient-specific devices. After collecting blood samples from the hospital, the PBS images are transmitted to the cloud server through a WiFi-enabled microscopic device. In the cloud server, a new fusion model that is capable of classifying ALL from PBS images is configured. The fusion model is trained using a dataset including 6512 original and segmented images from 89 individuals. Two input channels are used for the purpose of feature extraction in the fusion model. These channels include both the original and the segmented images. VGG16 is responsible for extracting features from the original images, whereas DenseNet-121 is responsible for extracting features from the segmented images. The two output features are merged together, and dense layers are used for the categorization of leukemia. The fusion model that has been suggested obtains an accuracy of 99.89%, a precision of 99.80%, and a recall of 99.72%, which places it in an excellent position for the categorization of leukemia. The proposed model outperformed several state-of-the-art Convolutional Neural Network (CNN) models in terms of performance. Consequently, this proposed model has the potential to save lives and effort. For a more comprehensive simulation of the entire methodology, a web application (Beta Version) has been developed in this study. This application is designed to determine the presence or absence of leukemia in individuals. The findings of this study hold significant potential for application in biomedical research, particularly in enhancing the accuracy of computer-aided leukemia detection.

Comparison of deep learning architectures for predicting amyloid positivity in Alzheimer's disease, mild cognitive impairment, and healthy aging, from T1-weighted brain structural MRI

Abnormal β-amyloid (Aβ) accumulation in the brain is an early indicator of Alzheimer's disease (AD) and is typically assessed through invasive procedures such as PET (positron emission tomography) or CSF (cerebrospinal fluid) assays. As new anti-Alzheimer's treatments can now successfully target amyloid pathology, there is a growing interest in predicting Aβ positivity (Aβ+) from less invasive, more widely available types of brain scans, such as T1-weighted (T1w) MRI. Here we compare multiple approaches to infer Aβ + from standard anatomical MRI: (1) classical machine learning algorithms, including logistic regression, XGBoost, and shallow artificial neural networks, (2) deep learning models based on 2D and 3D convolutional neural networks (CNNs), (3) a hybrid ANN-CNN, combining the strengths of shallow and deep neural networks, (4) transfer learning models based on CNNs, and (5) 3D Vision Transformers. All models were trained on paired MRI/PET data from 1,847 elderly participants (mean age: 75.1 yrs. ± 7.6SD; 863 females/984 males; 661 healthy controls, 889 with mild cognitive impairment (MCI), and 297 with Dementia), scanned as part of the Alzheimer's Disease Neuroimaging Initiative. We evaluated each model's balanced accuracy and F1 scores. While further tests on more diverse data are warranted, deep learning models trained on standard MRI showed promise for estimating Aβ + status, at least in people with MCI. This may offer a potential screening option before resorting to more invasive procedures.

Pulmonary nodule detection in low dose computed tomography using a medical-to-medical transfer learning approach

Purpose

Lung cancer is the second most common cancer and the leading cause of cancer death globally. Low dose computed tomography (LDCT) is the recommended imaging screening tool for the early detection of lung cancer. A fully automated computer-aided detection method for LDCT will greatly improve the existing clinical workflow. Most of the existing methods for lung detection are designed for high-dose CTs (HDCTs), and those methods cannot be directly applied to LDCTs due to domain shifts and inferior quality of LDCT images. In this work, we describe a semi-automated transfer learning-based approach for the early detection of lung nodules using LDCTs.

Approach

In this work, we developed an algorithm based on the object detection model, you only look once (YOLO) to detect lung nodules. The YOLO model was first trained on CTs, and the pre-trained weights were used as initial weights during the retraining of the model on LDCTs using a medical-to-medical transfer learning approach. The dataset for this study was from a screening trial consisting of LDCTs acquired from 50 biopsy-confirmed lung cancer patients obtained over 3 consecutive years (T1, T2, and T3). About 60 lung cancer patients' HDCTs were obtained from a public dataset. The developed model was evaluated using a hold-out test set comprising 15 patient cases (93 slices with cancerous nodules) using precision, specificity, recall, and F1-score. The evaluation metrics were reported patient-wise on a per-year basis and averaged for 3 years. For comparative analysis, the proposed detection model was trained using pre-trained weights from the COCO dataset as the initial weights. A paired t-test and chi-squared test with an alpha value of 0.05 were used for statistical significance testing.

Results

The results were reported by comparing the proposed model developed using HDCT pre-trained weights with COCO pre-trained weights. The former approach versus the latter approach obtained a precision of 0.982 versus 0.93 in detecting cancerous nodules, specificity of 0.923 versus 0.849 in identifying slices with no cancerous nodules, recall of 0.87 versus 0.886, and F1-score of 0.924 versus 0.903. As the nodule progressed, the former approach achieved a precision of 1, specificity of 0.92, and sensitivity of 0.930. The statistical analysis performed in the comparative study resulted in a p -value of 0.0054 for precision and a p -value of 0.00034 for specificity.

Conclusions

In this study, a semi-automated method was developed to detect lung nodules in LDCTs using HDCT pre-trained weights as the initial weights and retraining the model. Further, the results were compared by replacing HDCT pre-trained weights in the above approach with COCO pre-trained weights. The proposed method may identify early lung nodules during the screening program, reduce overdiagnosis and follow-ups due to misdiagnosis in LDCTs, start treatment options in the affected patients, and lower the mortality rate.

A practical guide to the implementation of artificial intelligence in orthopaedic research-Part 2: A technical introduction

Recent advances in artificial intelligence (AI) present a broad range of possibilities in medical research. However, orthopaedic researchers aiming to participate in research projects implementing AI-based techniques require a sound understanding of the technical fundamentals of this rapidly developing field. Initial sections of this technical primer provide an overview of the general and the more detailed taxonomy of AI methods. Researchers are presented with the technical basics of the most frequently performed machine learning (ML) tasks, such as classification, regression, clustering and dimensionality reduction. Additionally, the spectrum of supervision in ML including the domains of supervised, unsupervised, semisupervised and self-supervised learning will be explored. Recent advances in neural networks (NNs) and deep learning (DL) architectures have rendered them essential tools for the analysis of complex medical data, which warrants a rudimentary technical introduction to orthopaedic researchers. Furthermore, the capability of natural language processing (NLP) to interpret patterns in human language is discussed and may offer several potential applications in medical text classification, patient sentiment analysis and clinical decision support. The technical discussion concludes with the transformative potential of generative AI and large language models (LLMs) on AI research. Consequently, this second article of the series aims to equip orthopaedic researchers with the fundamental technical knowledge required to engage in interdisciplinary collaboration in AI-driven orthopaedic research.

LEVEL OF EVIDENCE

Level IV.

EffiSegNet: Gastrointestinal Polyp Segmentation through a Pre-Trained EfficientNet-based Network with a Simplified Decoder

This work introduces EffiSegNet, a novel segmentation framework leveraging transfer learning with a pre-trained Convolutional Neural Network (CNN) classifier as its backbone. Deviating from traditional architectures with a symmetric U-shape, EffiSegNet simplifies the decoder and utilizes full-scale feature fusion to minimize computational cost and the number of parameters. We evaluated our model on the gastrointestinal polyp segmentation task using the publicly available Kvasir-SEG dataset, achieving state-of-the-art results. Specifically, the EffiSegNet-B4 network variant achieved an F 1 score of 0.9552, mean Dice (mDice) 0.9483, mean Intersection over Union (mIoU) 0.9056, Precision 0.9679, and Recall 0.9429 with a pre-trained backbone - to the best of our knowledge, the highest reported scores in the literature for this dataset. Additional training from scratch also demonstrated exceptional performance compared to previous work, achieving an F 1 score of 0.9286, mDice 0.9207, mIoU 0.8668, Precision 0.9311 and Recall 0.9262. These results underscore the importance of a well-designed encoder in image segmentation networks and the effectiveness of transfer learning approaches.

Imaging in pleural Mesothelioma: A review of the 16th International Conference of the International Mesothelioma Interest Group

Imaging continues to gain a greater role in the assessment and clinical management of patients with mesothelioma. This communication summarizes the oral presentations from the imaging session at the 2023 International Conference of the International Mesothelioma Interest Group (iMig), which was held in Lille, France from June 26 to 28, 2023. Topics at this session included an overview of best practices for clinical imaging of mesothelioma as reported by an iMig consensus panel, emerging imaging techniques for surgical planning, radiologic assessment of malignant pleural effusion, a radiomics-based transfer learning model to predict patient response to treatment, automated assessment of early contrast enhancement, and tumor thickness for response assessment in peritoneal mesothelioma.

Development of oculomics artificial intelligence for cardiovascular risk factors: A case study in fundus oculomics for HbA1c assessment and clinically relevant considerations for clinicians

Artificial Intelligence (AI) is transforming healthcare, notably in ophthalmology, where its ability to interpret images and data can significantly enhance disease diagnosis and patient care. Recent developments in oculomics, the integration of ophthalmic features to develop biomarkers for systemic diseases, have demonstrated the potential for providing rapid, non-invasive methods of screening leading to enhance in early detection and improve healthcare quality, particularly in underserved areas. However, the widespread adoption of such AI-based technologies faces challenges primarily related to the trustworthiness of the system. We demonstrate the potential and considerations needed to develop trustworthy AI in oculomics through a pilot study for HbA1c assessment using an AI-based approach. We then discuss various challenges, considerations, and solutions that have been developed for powerful AI technologies in the past in healthcare and subsequently apply these considerations to the oculomics pilot study. Building upon the observations in the study we highlight the challenges and opportunities for advancing trustworthy AI in oculomics. Ultimately, oculomics presents as a powerful and emerging technology in ophthalmology and understanding how to optimize transparency prior to clinical adoption is of utmost importance.

Overcoming data scarcity in biomedical imaging with a foundational multi-task model

Foundational models, pretrained on a large scale, have demonstrated substantial success across non-medical domains. However, training these models typically requires large, comprehensive datasets, which contrasts with the smaller and more specialized datasets common in biomedical imaging. Here we propose a multi-task learning strategy that decouples the number of training tasks from memory requirements. We trained a universal biomedical pretrained model (UMedPT) on a multi-task database including tomographic, microscopic and X-ray images, with various labeling strategies such as classification, segmentation and object detection. The UMedPT foundational model outperformed ImageNet pretraining and previous state-of-the-art models. For classification tasks related to the pretraining database, it maintained its performance with only 1% of the original training data and without fine-tuning. For out-of-domain tasks it required only 50% of the original training data. In an external independent validation, imaging features extracted using UMedPT proved to set a new standard for cross-center transferability.

Multi-modal deep learning from imaging genomic data for schizophrenia classification

Background

Schizophrenia (SZ) is a psychiatric condition that adversely affects an individual's cognitive, emotional, and behavioral aspects. The etiology of SZ, although extensively studied, remains unclear, as multiple factors come together to contribute toward its development. There is a consistent body of evidence documenting the presence of structural and functional deviations in the brains of individuals with SZ. Moreover, the hereditary aspect of SZ is supported by the significant involvement of genomics markers. Therefore, the need to investigate SZ from a multi-modal perspective and develop approaches for improved detection arises.

Methods

Our proposed method employed a deep learning framework combining features from structural magnetic resonance imaging (sMRI), functional magnetic resonance imaging (fMRI), and genetic markers such as single nucleotide polymorphism (SNP). For sMRI, we used a pre-trained DenseNet to extract the morphological features. To identify the most relevant functional connections in fMRI and SNPs linked to SZ, we applied a 1-dimensional convolutional neural network (CNN) followed by layerwise relevance propagation (LRP). Finally, we concatenated these obtained features across modalities and fed them to the extreme gradient boosting (XGBoost) tree-based classifier to classify SZ from healthy control (HC).

Results

Experimental evaluation on clinical dataset demonstrated that, compared to the outcomes obtained from each modality individually, our proposed multi-modal approach performed classification of SZ individuals from HC with an improved accuracy of 79.01%.

Conclusion

We proposed a deep learning based framework that selects multi-modal (sMRI, fMRI and genetic) features efficiently and fuse them to obtain improved classification scores. Additionally, by using Explainable AI (XAI), we were able to pinpoint and validate significant functional network connections and SNPs that contributed the most toward SZ classification, providing necessary interpretation behind our findings.

Advancements in computer vision and pathology: Unraveling the potential of artificial intelligence for precision diagnosis and beyond

The integration of computer vision into pathology through slide digitalization represents a transformative leap in the field's evolution. Traditional pathology methods, while reliable, are often time-consuming and susceptible to intra- and interobserver variability. In contrast, computer vision, empowered by artificial intelligence (AI) and machine learning (ML), promises revolutionary changes, offering consistent, reproducible, and objective results with ever-increasing speed and scalability. The applications of advanced algorithms and deep learning architectures like CNNs and U-Nets augment pathologists' diagnostic capabilities, opening new frontiers in automated image analysis. As these technologies mature and integrate into digital pathology workflows, they are poised to provide deeper insights into disease processes, quantify and standardize biomarkers, enhance patient outcomes, and automate routine tasks, reducing pathologists' workload. However, this transformative force calls for cross-disciplinary collaboration between pathologists, computer scientists, and industry innovators to drive research and development. While acknowledging its potential, this chapter addresses the limitations of AI in pathology, encompassing technical, practical, and ethical considerations during development and implementation.

Assessing CT-based Volumetric Analysis via Transfer Learning with MRI and Manual Labels for Idiopathic Normal Pressure Hydrocephalus

Background

Brain computed tomography (CT) is an accessible and commonly utilized technique for assessing brain structure. In cases of idiopathic normal pressure hydrocephalus (iNPH), the presence of ventriculomegaly is often neuroradiologically evaluated by visual rating and manually measuring each image. Previously, we have developed and tested a deep-learning-model that utilizes transfer learning from magnetic resonance imaging (MRI) for CT-based intracranial tissue segmentation. Accordingly, herein we aimed to enhance the segmentation of ventricular cerebrospinal fluid (VCSF) in brain CT scans and assess the performance of automated brain CT volumetrics in iNPH patient diagnostics.

Methods

The development of the model used a two-stage approach. Initially, a 2D U-Net model was trained to predict VCSF segmentations from CT scans, using paired MR-VCSF labels from healthy controls. This model was subsequently refined by incorporating manually segmented lateral CT-VCSF labels from iNPH patients, building on the features learned from the initial U-Net model. The training dataset included 734 CT datasets from healthy controls paired with T1-weighted MRI scans from the Gothenburg H70 Birth Cohort Studies and 62 CT scans from iNPH patients at Uppsala University Hospital. To validate the model's performance across diverse patient populations, external clinical images including scans of 11 iNPH patients from the Universitatsmedizin Rostock, Germany, and 30 iNPH patients from the University of Alabama at Birmingham, United States were used. Further, we obtained three CT-based volumetric measures (CTVMs) related to iNPH.

Results

Our analyses demonstrated strong volumetric correlations (ϱ=0.91, p<0.001) between automatically and manually derived CT-VCSF measurements in iNPH patients. The CTVMs exhibited high accuracy in differentiating iNPH patients from controls in external clinical datasets with an AUC of 0.97 and in the Uppsala University Hospital datasets with an AUC of 0.99.

Discussion

CTVMs derived through deep learning, show potential for assessing and quantifying morphological features in hydrocephalus. Critically, these measures performed comparably to gold-standard neuroradiology assessments in distinguishing iNPH from healthy controls, even in the presence of intraventricular shunt catheters. Accordingly, such an approach may serve to improve the radiological evaluation of iNPH diagnosis/monitoring (i.e., treatment responses). Since CT is much more widely available than MRI, our results have considerable clinical impact.

Knowledge Transfer from Macro-world to Micro-world: Enhancing 3D Cryo-ET Classification through Fine-Tuning Video-based Deep Models

MOTIVATION

Deep learning models have achieved remarkable success in a wide range of natural-world tasks, such as vision, language, and speech recognition. These accomplishments are largely attributed to the availability of open-source large-scale datasets. More importantly, pre-trained foundational modellearnings exhibit a surprising degree of transferability to downstream tasks, enabling efficient learning even with limited training examples. However, the application of such natural-domain models to the domain of tiny Cryo-Electron Tomography (Cryo-ET) images has been a relatively unexplored frontier. This research is motivated by the intuition that 3D Cryo-ET voxel data can be conceptually viewed as a sequence of progressively evolving video frames.

RESULTS

Leveraging the above insight, we propose a novel approach that involves the utilization of 3D models pre-trained on large-scale video datasets to enhance Cryo-ET subtomogram classification. Our experiments, conducted on both simulated and real Cryo-ET datasets, reveal compelling results. The use of video initialization not only demonstrates improvements in classification accuracy but also substantially reduces training costs. Further analyses provide additional evidence of the value of video initialization in enhancing subtomogram feature extraction. Additionally, we observe that video initialization yields similar positive effects when applied to medical 3D classification tasks, underscoring the potential of cross-domain knowledge transfer from video-based models to advance the state-of-the-art in a wide range of biological and medical data types.

AVAILABILITY AND IMPLEMENTATION

https://github.com/xulabs/aitom.

Preliminary study on AI-assisted diagnosis of bone remodeling in chronic maxillary sinusitis

OBJECTIVE

To construct the deep learning convolution neural network (CNN) model and machine learning support vector machine (SVM) model of bone remodeling of chronic maxillary sinusitis (CMS) based on CT image data to improve the accuracy of image diagnosis.

METHODS

Maxillary sinus CT data of 1000 samples in 500 patients from January 2018 to December 2021 in our hospital was collected. The first part is the establishment and testing of chronic maxillary sinusitis detection model by 461 images. The second part is the establishment and testing of the detection model of chronic maxillary sinusitis with bone remodeling by 802 images. The sensitivity, specificity and accuracy and area under the curve (AUC) value of the test set were recorded, respectively.

RESULTS

Preliminary application results of CT based AI in the diagnosis of chronic maxillary sinusitis and bone remodeling. The sensitivity, specificity and accuracy of the test set of 93 samples of CMS, were 0.9796, 0.8636 and 0.9247, respectively. Simultaneously, the value of AUC was 0.94. And the sensitivity, specificity and accuracy of the test set of 161 samples of CMS with bone remodeling were 0.7353, 0.9685 and 0.9193, respectively. Simultaneously, the value of AUC was 0.89.

CONCLUSION

It is feasible to use artificial intelligence research methods such as deep learning and machine learning to automatically identify CMS and bone remodeling in MSCT images of paranasal sinuses, which is helpful to standardize imaging diagnosis and meet the needs of clinical application.

A comprehensive approach for osteoporosis detection through chest CT analysis and bone turnover markers: harnessing radiomics and deep learning techniques

Purpose

The main objective of this study is to assess the possibility of using radiomics, deep learning, and transfer learning methods for the analysis of chest CT scans. An additional aim is to combine these techniques with bone turnover markers to identify and screen for osteoporosis in patients.

Method

A total of 488 patients who had undergone chest CT and bone turnover marker testing, and had known bone mineral density, were included in this study. ITK-SNAP software was used to delineate regions of interest, while radiomics features were extracted using Python. Multiple 2D and 3D deep learning models were trained to identify these regions of interest. The effectiveness of these techniques in screening for osteoporosis in patients was compared.

Result

Clinical models based on gender, age, and β-cross achieved an accuracy of 0.698 and an AUC of 0.665. Radiomics models, which utilized 14 selected radiomics features, achieved a maximum accuracy of 0.750 and an AUC of 0.739. The test group yielded promising results: the 2D Deep Learning model achieved an accuracy of 0.812 and an AUC of 0.855, while the 3D Deep Learning model performed even better with an accuracy of 0.854 and an AUC of 0.906. Similarly, the 2D Transfer Learning model achieved an accuracy of 0.854 and an AUC of 0.880, whereas the 3D Transfer Learning model exhibited an accuracy of 0.740 and an AUC of 0.737. Overall, the application of 3D deep learning and 2D transfer learning techniques on chest CT scans showed excellent screening performance in the context of osteoporosis.

Conclusion

Bone turnover markers may not be necessary for osteoporosis screening, as 3D deep learning and 2D transfer learning techniques utilizing chest CT scans proved to be equally effective alternatives.

Automating Linear and Angular Measurements for the Hip and Knee After Computed Tomography: Validation of a Three-Stage Deep Learning and Computer Vision-Based Pipeline for Pathoanatomic Assessment

Background

Variability in the bony morphology of pathologic hips/knees is a challenge in automating preoperative computed tomography (CT) scan measurements. With the increasing prevalence of CT for advanced preoperative planning, processing this data represents a critical bottleneck in presurgical planning, research, and development. The purpose of this study was to demonstrate a reproducible and scalable methodology for analyzing CT-based anatomy to process hip and knee anatomy for perioperative planning and execution.

Methods

One hundred patients with preoperative CT scans undergoing total knee arthroplasty for osteoarthritis were processed. A two-step deep learning pipeline of classification and segmentation models was developed that identifies landmark images and then generates contour representations. We utilized an open-source computer vision library to compute measurements. Classification models were assessed by accuracy, precision, and recall. Segmentation models were evaluated using dice and mean Intersection over Union (IOU) metrics. Contour measurements were compared against manual measurements to validate posterior condylar axis angle, sulcus angle, trochlear groove-tibial tuberosity distance, acetabular anteversion, and femoral version.

Results

Classifiers identified landmark images with accuracy of 0.91 and 0.88 for hip and knee models, respectively. Segmentation models demonstrated mean IOU scores above 0.95 with the highest dice coefficient of 0.957 [0.954-0.961] (UNet3+) and the highest mean IOU of 0.965 [0.961-0.969] (Attention U-Net). There were no statistically significant differences for the measurements taken automatically vs manually (P > 0.05). Average time for the pipeline to preprocess (48.65 +/- 4.41 sec), classify/retrieve landmark images (8.36 +/- 3.40 sec), segment images (<1 sec), and obtain measurements was 2.58 (+/- 1.92) minutes.

Conclusions

A fully automated three-stage deep learning and computer vision-based pipeline of classification and segmentation models accurately localized, segmented, and measured landmark hip and knee images for patients undergoing total knee arthroplasty. Incorporation of clinical parameters, like patient-reported outcome measures and instability risk, will be important considerations alongside anatomic parameters.

VER-Net: a hybrid transfer learning model for lung cancer detection using CT scan images

BACKGROUND

Lung cancer is the second most common cancer worldwide, with over two million new cases per year. Early identification would allow healthcare practitioners to handle it more effectively. The advancement of computer-aided detection systems significantly impacted clinical analysis and decision-making on human disease. Towards this, machine learning and deep learning techniques are successfully being applied. Due to several advantages, transfer learning has become popular for disease detection based on image data.

METHODS

In this work, we build a novel transfer learning model (VER-Net) by stacking three different transfer learning models to detect lung cancer using lung CT scan images. The model is trained to map the CT scan images with four lung cancer classes. Various measures, such as image preprocessing, data augmentation, and hyperparameter tuning, are taken to improve the efficacy of VER-Net. All the models are trained and evaluated using multiclass classifications chest CT images.

RESULTS

The experimental results confirm that VER-Net outperformed the other eight transfer learning models compared with. VER-Net scored 91%, 92%, 91%, and 91.3% when tested for accuracy, precision, recall, and F1-score, respectively. Compared to the state-of-the-art, VER-Net has better accuracy.

CONCLUSION

VER-Net is not only effectively used for lung cancer detection but may also be useful for other diseases for which CT scan images are available.

BarlowTwins-CXR: enhancing chest X-ray abnormality localization in heterogeneous data with cross-domain self-supervised learning

BACKGROUND

Chest X-ray imaging based abnormality localization, essential in diagnosing various diseases, faces significant clinical challenges due to complex interpretations and the growing workload of radiologists. While recent advances in deep learning offer promising solutions, there is still a critical issue of domain inconsistency in cross-domain transfer learning, which hampers the efficiency and accuracy of diagnostic processes. This study aims to address the domain inconsistency problem and improve autonomic abnormality localization performance of heterogeneous chest X-ray image analysis, particularly in detecting abnormalities, by developing a self-supervised learning strategy called "BarlwoTwins-CXR".

METHODS

We utilized two publicly available datasets: the NIH Chest X-ray Dataset and the VinDr-CXR. The BarlowTwins-CXR approach was conducted in a two-stage training process. Initially, self-supervised pre-training was performed using an adjusted Barlow Twins algorithm on the NIH dataset with a Resnet50 backbone pre-trained on ImageNet. This was followed by supervised fine-tuning on the VinDr-CXR dataset using Faster R-CNN with Feature Pyramid Network (FPN). The study employed mean Average Precision (mAP) at an Intersection over Union (IoU) of 50% and Area Under the Curve (AUC) for performance evaluation.

RESULTS

Our experiments showed a significant improvement in model performance with BarlowTwins-CXR. The approach achieved a 3% increase in mAP50 accuracy compared to traditional ImageNet pre-trained models. In addition, the Ablation CAM method revealed enhanced precision in localizing chest abnormalities. The study involved 112,120 images from the NIH dataset and 18,000 images from the VinDr-CXR dataset, indicating robust training and testing samples.

CONCLUSION

BarlowTwins-CXR significantly enhances the efficiency and accuracy of chest X-ray image-based abnormality localization, outperforming traditional transfer learning methods and effectively overcoming domain inconsistency in cross-domain scenarios. Our experiment results demonstrate the potential of using self-supervised learning to improve the generalizability of models in medical settings with limited amounts of heterogeneous data. This approach can be instrumental in aiding radiologists, particularly in high-workload environments, offering a promising direction for future AI-driven healthcare solutions.

SPECT-MPI for Coronary Artery Disease: A Deep Learning Approach

Background

Worldwide, coronary artery disease (CAD) is a leading cause of mortality and morbidity and remains to be a top health priority in many countries. A non-invasive imaging modality for diagnosis of CAD such as single photon emission computed tomography-myocardial perfusion imaging (SPECT-MPI) is usually requested by cardiologists as it displays radiotracer distribution in the heart reflecting myocardial perfusion. The interpretation of SPECT-MPI is done visually by a nuclear medicine physician and is largely dependent on his clinical experience and showing significant inter-observer variability.

Objective

The aim of the study is to apply a deep learning approach in the classification of SPECT-MPI for perfusion abnormalities using convolutional neural networks (CNN).

Methods

A publicly available anonymized SPECT-MPI from a machine learning repository (https://www.kaggle.com/selcankaplan/spect-mpi) was used in this study involving 192 patients who underwent stress-test-rest Tc99m MPI. An exploratory approach of CNN hyperparameter selection to search for optimum neural network model was utilized with particular focus on various dropouts (0.2, 0.5, 0.7), batch sizes (8, 16, 32, 64), and number of dense nodes (32, 64, 128, 256). The base CNN model was also compared with the commonly used pre-trained CNNs in medical images such as VGG16, InceptionV3, DenseNet121 and ResNet50. All simulations experiments were performed in Kaggle using TensorFlow 2.6.0., Keras 2.6.0, and Python language 3.7.10.

Results

The best performing base CNN model with parameters consisting of 0.7 dropout, batch size 8, and 32 dense nodes generated the highest normalized Matthews Correlation Coefficient at 0.909 and obtained 93.75% accuracy, 96.00% sensitivity, 96.00% precision, and 96.00% F1-score. It also obtained higher classification performance as compared to the pre-trained architectures.

Conclusions

The results suggest that deep learning approaches through the use of CNN models can be deployed by nuclear medicine physicians in their clinical practice to further augment their decision skills in the interpretation of SPECT-MPI tests. These CNN models can also be used as a dependable and valid second opinion that can aid physicians as a decision-support tool as well as serve as teaching or learning materials for the less-experienced physicians particularly those still in their training career. These highlights the clinical utility of deep learning approaches through CNN models in the practice of nuclear cardiology.

Automated medication verification system (AMVS): System based on edge detection and CNN classification drug on embedded systems

A novel automated medication verification system (AMVS) aims to address the limitation of manual medication verification among healthcare professionals with a high workload, thereby reducing medication errors in hospitals. Specifically, the manual medication verification process is time-consuming and prone to errors, especially in healthcare settings with high workloads. The proposed system strategy is to streamline and automate this process, enhancing efficiency and reducing medication errors. The system employs deep learning models to swiftly and accurately classify multiple medications within a single image without requiring manual labeling during model construction. It comprises edge detection and classification to verify medication types. Unlike previous studies conducted in open spaces, our study takes place in a closed space to minimize the impact of optical changes on image capture. During the experimental process, the system individually identifies each drug within the image by edge detection method and utilizes a classification model to determine each drug type. Our research has successfully developed a fully automated drug recognition system, achieving an accuracy of over 95 % in identifying drug types and conducting segmentation analyses. Specifically, the system demonstrates an accuracy rate of approximately 96 % for drug sets containing fewer than ten types and 93 % for those with ten types. This verification system builds an image classification model quickly. It holds promising potential in assisting nursing staff during AMVS, thereby reducing the likelihood of medication errors and alleviating the burden on nursing staff.

Cross-site validation of lung cancer diagnosis by electronic nose with deep learning: a multicenter prospective study

BACKGROUND

Although electronic nose (eNose) has been intensively investigated for diagnosing lung cancer, cross-site validation remains a major obstacle to be overcome and no studies have yet been performed.

METHODS

Patients with lung cancer, as well as healthy control and diseased control groups, were prospectively recruited from two referral centers between 2019 and 2022. Deep learning models for detecting lung cancer with eNose breathprint were developed using training cohort from one site and then tested on cohort from the other site. Semi-Supervised Domain-Generalized (Semi-DG) Augmentation (SDA) and Noise-Shift Augmentation (NSA) methods with or without fine-tuning was applied to improve performance.

RESULTS

In this study, 231 participants were enrolled, comprising a training/validation cohort of 168 individuals (90 with lung cancer, 16 healthy controls, and 62 diseased controls) and a test cohort of 63 individuals (28 with lung cancer, 10 healthy controls, and 25 diseased controls). The model has satisfactory results in the validation cohort from the same hospital while directly applying the trained model to the test cohort yielded suboptimal results (AUC, 0.61, 95% CI: 0.47─0.76). The performance improved after applying data augmentation methods in the training cohort (SDA, AUC: 0.89 [0.81─0.97]; NSA, AUC:0.90 [0.89─1.00]). Additionally, after applying fine-tuning methods, the performance further improved (SDA plus fine-tuning, AUC:0.95 [0.89─1.00]; NSA plus fine-tuning, AUC:0.95 [0.90─1.00]).

CONCLUSION

Our study revealed that deep learning models developed for eNose breathprint can achieve cross-site validation with data augmentation and fine-tuning. Accordingly, eNose breathprints emerge as a convenient, non-invasive, and potentially generalizable solution for lung cancer detection.

CLINICAL TRIAL REGISTRATION

This study is not a clinical trial and was therefore not registered.

Applications of deep learning in trauma radiology: A narrative review

Diagnostic imaging is essential in modern trauma care for initial evaluation and identifying injuries requiring intervention. Deep learning (DL) has become mainstream in medical image analysis and has shown promising efficacy for classification, segmentation, and lesion detection. This narrative review provides the fundamental concepts for developing DL algorithms in trauma imaging and presents an overview of current progress in each modality. DL has been applied to detect free fluid on Focused Assessment with Sonography for Trauma (FAST), traumatic findings on chest and pelvic X-rays, and computed tomography (CT) scans, identify intracranial hemorrhage on head CT, detect vertebral fractures, and identify injuries to organs like the spleen, liver, and lungs on abdominal and chest CT. Future directions involve expanding dataset size and diversity through federated learning, enhancing model explainability and transparency to build clinician trust, and integrating multimodal data to provide more meaningful insights into traumatic injuries. Though some commercial artificial intelligence products are Food and Drug Administration-approved for clinical use in the trauma field, adoption remains limited, highlighting the need for multi-disciplinary teams to engineer practical, real-world solutions. Overall, DL shows immense potential to improve the efficiency and accuracy of trauma imaging, but thoughtful development and validation are critical to ensure these technologies positively impact patient care.

Deep Transfer Learning Using Real-World Image Features for Medical Image Classification, with a Case Study on Pneumonia X-ray Images

Deep learning has profoundly influenced various domains, particularly medical image analysis. Traditional transfer learning approaches in this field rely on models pretrained on domain-specific medical datasets, which limits their generalizability and accessibility. In this study, we propose a novel framework called real-world feature transfer learning, which utilizes backbone models initially trained on large-scale general-purpose datasets such as ImageNet. We evaluate the effectiveness and robustness of this approach compared to models trained from scratch, focusing on the task of classifying pneumonia in X-ray images. Our experiments, which included converting grayscale images to RGB format, demonstrate that real-world-feature transfer learning consistently outperforms conventional training approaches across various performance metrics. This advancement has the potential to accelerate deep learning applications in medical imaging by leveraging the rich feature representations learned from general-purpose pretrained models. The proposed methodology overcomes the limitations of domain-specific pretrained models, thereby enabling accelerated innovation in medical diagnostics and healthcare. From a mathematical perspective, we formalize the concept of real-world feature transfer learning and provide a rigorous mathematical formulation of the problem. Our experimental results provide empirical evidence supporting the effectiveness of this approach, laying the foundation for further theoretical analysis and exploration. This work contributes to the broader understanding of feature transferability across domains and has significant implications for the development of accurate and efficient models for medical image analysis, even in resource-constrained settings.

Convolutional neural network deep learning model accurately detects rectal cancer in endoanal ultrasounds

BACKGROUND

Imaging is vital for assessing rectal cancer, with endoanal ultrasound (EAUS) being highly accurate in large tertiary medical centers. However, EAUS accuracy drops outside such settings, possibly due to varied examiner experience and fewer examinations. This underscores the need for an AI-based system to enhance accuracy in non-specialized centers. This study aimed to develop and validate deep learning (DL) models to differentiate rectal cancer in standard EAUS images.

METHODS

A transfer learning approach with fine-tuned DL architectures was employed, utilizing a dataset of 294 images. The performance of DL models was assessed through a tenfold cross-validation.

RESULTS

The DL diagnostics model exhibited a sensitivity and accuracy of 0.78 each. In the identification phase, the automatic diagnostic platform achieved an area under the curve performance of 0.85 for diagnosing rectal cancer.

CONCLUSIONS

This research demonstrates the potential of DL models in enhancing rectal cancer detection during EAUS, especially in settings with lower examiner experience. The achieved sensitivity and accuracy suggest the viability of incorporating AI support for improved diagnostic outcomes in non-specialized medical centers.

Just how transformative will AI/ML be for immuno-oncology?

Immuno-oncology involves the study of approaches which harness the patient's immune system to fight malignancies. Immuno-oncology, as with every other biomedical and clinical research field as well as clinical operations, is in the midst of technological revolutions, which vastly increase the amount of available data. Recent advances in artificial intelligence and machine learning (AI/ML) have received much attention in terms of their potential to harness available data to improve insights and outcomes in many areas including immuno-oncology. In this review, we discuss important aspects to consider when evaluating the potential impact of AI/ML applications in the clinic. We highlight four clinical/biomedical challenges relevant to immuno-oncology and how they may be able to be addressed by the latest advancements in AI/ML. These challenges include (1) efficiency in clinical workflows, (2) curation of high-quality image data, (3) finding, extracting and synthesizing text knowledge as well as addressing, and (4) small cohort size in immunotherapeutic evaluation cohorts. Finally, we outline how advancements in reinforcement and federated learning, as well as the development of best practices for ethical and unbiased data generation, are likely to drive future innovations.

Trustworthy deep learning framework for the detection of abnormalities in X-ray shoulder images

Musculoskeletal conditions affect an estimated 1.7 billion people worldwide, causing intense pain and disability. These conditions lead to 30 million emergency room visits yearly, and the numbers are only increasing. However, diagnosing musculoskeletal issues can be challenging, especially in emergencies where quick decisions are necessary. Deep learning (DL) has shown promise in various medical applications. However, previous methods had poor performance and a lack of transparency in detecting shoulder abnormalities on X-ray images due to a lack of training data and better representation of features. This often resulted in overfitting, poor generalisation, and potential bias in decision-making. To address these issues, a new trustworthy DL framework has been proposed to detect shoulder abnormalities (such as fractures, deformities, and arthritis) using X-ray images. The framework consists of two parts: same-domain transfer learning (TL) to mitigate imageNet mismatch and feature fusion to reduce error rates and improve trust in the final result. Same-domain TL involves training pre-trained models on a large number of labelled X-ray images from various body parts and fine-tuning them on the target dataset of shoulder X-ray images. Feature fusion combines the extracted features with seven DL models to train several ML classifiers. The proposed framework achieved an excellent accuracy rate of 99.2%, F1Score of 99.2%, and Cohen's kappa of 98.5%. Furthermore, the accuracy of the results was validated using three visualisation tools, including gradient-based class activation heat map (Grad CAM), activation visualisation, and locally interpretable model-independent explanations (LIME). The proposed framework outperformed previous DL methods and three orthopaedic surgeons invited to classify the test set, who obtained an average accuracy of 79.1%. The proposed framework has proven effective and robust, improving generalisation and increasing trust in the final results.

High Resolution Multi-delay Arterial Spin Labeling with Transformer based Denoising for Pediatric Perfusion MRI

Multi-delay arterial spin labeling (MDASL) can quantitatively measure cerebral blood flow (CBF) and arterial transit time (ATT), which is particularly suitable for pediatric perfusion imaging. Here we present a high resolution (iso-2mm) MDASL protocol and performed test-retest scans on 21 typically developing children aged 8 to 17 years. We further proposed a Transformer-based deep learning (DL) model with k-space weighted image average (KWIA) denoised images as reference for training the model. The performance of the model was evaluated by the SNR of perfusion images, as well as the SNR, bias and repeatability of the fitted CBF and ATT maps. The proposed method was compared to several benchmark methods including KWIA, joint denoising and reconstruction with total generalized variation (TGV) regularization, as well as directly applying a pretrained Transformer model on a larger dataset. The results show that the proposed Transformer model with KWIA reference can effectively denoise multi-delay ASL images, not only improving the SNR for perfusion images of each delay, but also improving the SNR for the fitted CBF and ATT maps. The proposed method also improved test-retest repeatability of whole-brain perfusion measurements. This may facilitate the use of MDASL in neurodevelopmental studies to characterize typical and aberrant brain development.

Transfer learning-based PET/CT three-dimensional convolutional neural network fusion of image and clinical information for prediction of EGFR mutation in lung adenocarcinoma

BACKGROUND

To introduce a three-dimensional convolutional neural network (3D CNN) leveraging transfer learning for fusing PET/CT images and clinical data to predict EGFR mutation status in lung adenocarcinoma (LADC).

METHODS

Retrospective data from 516 LADC patients, encompassing preoperative PET/CT images, clinical information, and EGFR mutation status, were divided into training (n = 404) and test sets (n = 112). Several deep learning models were developed utilizing transfer learning, involving CT-only and PET-only models. A dual-stream model fusing PET and CT and a three-stream transfer learning model (TS_TL) integrating clinical data were also developed. Image preprocessing includes semi-automatic segmentation, resampling, and image cropping. Considering the impact of class imbalance, the performance of the model was evaluated using ROC curves and AUC values.

RESULTS

TS_TL model demonstrated promising performance in predicting the EGFR mutation status, with an AUC of 0.883 (95%CI = 0.849-0.917) in the training set and 0.730 (95%CI = 0.629-0.830) in the independent test set. Particularly in advanced LADC, the model achieved an AUC of 0.871 (95%CI = 0.823-0.919) in the training set and 0.760 (95%CI = 0.638-0.881) in the test set. The model identified distinct activation areas in solid or subsolid lesions associated with wild and mutant types. Additionally, the patterns captured by the model were significantly altered by effective tyrosine kinase inhibitors treatment, leading to notable changes in predicted mutation probabilities.

CONCLUSION

PET/CT deep learning model can act as a tool for predicting EGFR mutation in LADC. Additionally, it offers clinicians insights for treatment decisions through evaluations both before and after treatment.

Leveraging artificial intelligence for perioperative cancer risk assessment of oral potentially malignant disorders

Oral potentially malignant disorders (OPMDs) are mucosal conditions with an inherent disposition to develop oral squamous cell carcinoma. Surgical management is the most preferred strategy to prevent malignant transformation in OPMDs, and surgical approaches to treatment include conventional scalpel excision, laser surgery, cryotherapy, and photodynamic therapy. However, in reality, since all patients with OPMDs will not develop oral squamous cell carcinoma in their lifetime, there is a need to stratify patients according to their risk of malignant transformation to streamline surgical intervention for patients with the highest risks. Artificial intelligence (AI) has the potential to integrate disparate factors influencing malignant transformation for robust, precise, and personalized cancer risk stratification of OPMD patients than current methods to determine the need for surgical resection, excision, or re-excision. Therefore, this article overviews existing AI models and tools, presents a clinical implementation pathway, and discusses necessary refinements to aid the clinical application of AI-based platforms for cancer risk stratification of OPMDs in surgical practice.

Classification in Early Fire Detection Using Multi-Sensor Nodes-A Transfer Learning Approach

Effective early fire detection is crucial for preventing damage to people and buildings, especially in fire-prone historic structures. However, due to the infrequent occurrence of fire events throughout a building's lifespan, real-world data for training models are often sparse. In this study, we applied feature representation transfer and instance transfer in the context of early fire detection using multi-sensor nodes. The goal was to investigate whether training data from a small-scale setup (source domain) can be used to identify various incipient fire scenarios in their early stages within a full-scale test room (target domain). In a first step, we employed Linear Discriminant Analysis (LDA) to create a new feature space solely based on the source domain data and predicted four different fire types (smoldering wood, smoldering cotton, smoldering cable and candle fire) in the target domain with a classification rate up to 69% and a Cohen's Kappa of 0.58. Notably, lower classification performance was observed for sensor node positions close to the wall in the full-scale test room. In a second experiment, we applied the TrAdaBoost algorithm as a common instance transfer technique to adapt the model to the target domain, assuming that sparse information from the target domain is available. Boosting the data from 1% to 30% was utilized for individual sensor node positions in the target domain to adapt the model to the target domain. We found that additional boosting improved the classification performance (average classification rate of 73% and an average Cohen's Kappa of 0.63). However, it was noted that excessively boosting the data could lead to overfitting to a specific sensor node position in the target domain, resulting in a reduction in the overall classification performance.