Transfer learning for medical images analyses: A survey

Utilizing Pseudo Color Image to Improve the Performance of Deep Transfer Learning-Based Computer-Aided Diagnosis Schemes in Breast Mass Classification

The purpose of this study is to investigate the impact of using morphological information in classifying suspicious breast lesions. The widespread use of deep transfer learning can significantly improve the performance of the mammogram based CADx schemes. However, digital mammograms are grayscale images, while deep learning models are typically optimized using the natural images containing three channels. Thus, it is needed to convert the grayscale mammograms into three channel images for the input of deep transfer models. This study aims to develop a novel pseudo color image generation method which utilizes the mass contour information to enhance the classification performance. Accordingly, a total of 830 breast cancer cases were retrospectively collected, which contains 310 benign and 520 malignant cases, respectively. For each case, a total of four regions of interest (ROI) are collected from the grayscale images captured for both the CC and MLO views of the two breasts. Meanwhile, a total of seven pseudo color image sets are generated as the input of the deep learning models, which are created through a combination of the original grayscale image, a histogram equalized image, a bilaterally filtered image, and a segmented mass. Accordingly, the output features from four identical pre-trained deep learning models are concatenated and then processed by a support vector machine-based classifier to generate the final benign/malignant labels. The performance of each image set was evaluated and compared. The results demonstrate that the pseudo color sets containing the manually segmented mass performed significantly better than all other pseudo color sets, which achieved an AUC (area under the ROC curve) up to 0.889 ± 0.012 and an overall accuracy up to 0.816 ± 0.020, respectively. At the same time, the performance improvement is also dependent on the accuracy of the mass segmentation. The results of this study support our hypothesis that adding accurately segmented mass contours can provide complementary information, thereby enhancing the performance of the deep transfer model in classifying suspicious breast lesions.

Machine learning for automated classification of lung collagen in a urethane-induced lung injury mouse model

Dysregulation of lung tissue collagen level plays a vital role in understanding how lung diseases progress. However, traditional scoring methods rely on manual histopathological examination introducing subjectivity and inconsistency into the assessment process. These methods are further hampered by inter-observer variability, lack of quantification, and their time-consuming nature. To mitigate these drawbacks, we propose a machine learning-driven framework for automated scoring of lung collagen content. Our study begins with the collection of a lung slide image dataset from adult female mice using second harmonic generation (SHG) microscopy. In our proposed approach, first, we manually extracted features based on the 46 statistical parameters of fibrillar collagen. Subsequently, we pre-processed the images and utilized a pre-trained VGG16 model to uncover hidden features from pre-processed images. We then combined both image and statistical features to train various machine learning and deep neural network models for classification tasks. We employed advanced unsupervised techniques like K-means, principal component analysis (PCA), t-distributed stochastic neighbour embedding (t-SNE), and uniform manifold approximation and projection (UMAP) to conduct thorough image analysis for lung collagen content. Also, the evaluation of the trained models using the collagen data includes both binary and multi-label classification to predict lung cancer in a urethane-induced mouse model. Experimental validation of our proposed approach demonstrates promising results. We obtained an average accuracy of 83% and an area under the receiver operating characteristic curve (ROC AUC) values of 0.96 through the use of a support vector machine (SVM) model for binary categorization tasks. For multi-label classification tasks, to quantify the structural alteration of collagen, we attained an average accuracy of 73% and ROC AUC values of 1.0, 0.38, 0.95, and 0.86 for control, baseline, treatment_1, and treatment_2 groups, respectively. Our findings provide significant potential for enhancing diagnostic accuracy, understanding disease mechanisms, and improving clinical practice using machine learning and deep learning models.

Construction and Validation of a General Medical Image Dataset for Pretraining

In the field of deep learning for medical image analysis, training models from scratch are often used and sometimes, transfer learning from pretrained parameters on ImageNet models is also adopted. However, there is no universally accepted medical image dataset specifically designed for pretraining models currently. The purpose of this study is to construct such a general dataset and validate its effectiveness on downstream medical imaging tasks, including classification and segmentation. In this work, we first build a medical image dataset by collecting several public medical image datasets (CPMID). And then, some pretrained models used for transfer learning are obtained based on CPMID. Various-complexity Resnet and the Vision Transformer network are used as the backbone architectures. In the tasks of classification and segmentation on three other datasets, we compared the experimental results of training from scratch, from the pretrained parameters on ImageNet, and from the pretrained parameters on CPMID. Accuracy, the area under the receiver operating characteristic curve, and class activation map are used as metrics for classification performance. Intersection over Union as the metric is for segmentation evaluation. Utilizing the pretrained parameters on the constructed dataset CPMID, we achieved the best classification accuracy, weighted accuracy, and ROC-AUC values on three validation datasets. Notably, the average classification accuracy outperformed ImageNet-based results by 4.30%, 8.86%, and 3.85% respectively. Furthermore, we achieved the optimal balanced outcome of performance and efficiency in both classification and segmentation tasks. The pretrained parameters on the proposed dataset CPMID are very effective for common tasks in medical image analysis such as classification and segmentation.

Deep orthogonal multi-wavelength fusion for tomogram-free diagnosis in diffuse optical imaging

Novel portable diffuse optical tomography (DOT) devices for breast cancer lesions hold great promise for non-invasive, non-ionizing breast cancer screening. Critical to this capability is not just the identification of lesions but rather the complex problem of discriminating between malignant and benign lesions. To accurately reconstruct the highly heterogeneous tissue of a cancer lesion in healthy breast tissue using DOT, multiple wavelengths can be leveraged to maximize signal penetration while minimizing sensitivity to noise. However, these wavelength responses can overlap, capture common information, and correlate, potentially confounding reconstruction and downstream end tasks. We show that an orthogonal fusion loss regularizes multi-wavelength DOT leading to improved reconstruction and accuracy of end-to-end discrimination of malignant versus benign lesions. We further show that our raw-to-task model significantly reduces computational complexity without sacrificing accuracy, making it ideal for real-time throughput, desired in medical settings where handheld devices have severely restricted power budgets. Furthermore, our results indicate that image reconstruction is not necessary for unbiased classification of lesions with a balanced accuracy of 77% and 66% on the synthetic dataset and clinical dataset, respectively, using the raw-to-task model. Code is available at https://github.com/sfu-mial/FuseNet.

A deep learning approach to direct immunofluorescence pattern recognition in autoimmune bullous diseases

BACKGROUND

Artificial intelligence (AI) is reshaping healthcare, using machine and deep learning (DL) to enhance disease management. Dermatology has seen improved diagnostics, particularly in skin cancer detection, through the integration of AI. However, the potential of AI in automating immunofluorescence imaging for autoimmune bullous skin diseases (AIBDs) remains untapped. While direct immunofluorescence (DIF) supports diagnosis, its manual interpretation can hinder efficiency. The use of DL to classify DIF patterns automatically, including the intercellular (ICP) and linear pattern (LP), holds promise for improving the diagnosis of AIBDs.

OBJECTIVES

To develop AI algorithms for automated classification of AIBD DIF patterns, such as ICP and LP, in order to enhance diagnostic accuracy, streamline disease management and improve patient outcomes through DL-driven immunofluorescence interpretation.

METHODS

We collected immunofluorescence images from skin biopsies of patients suspected of having an AIBD between January 2022 and January 2024. Skin tissue was obtained via a 5-mm punch biopsy, prepared for DIF. Experienced dermatologists classified the images as ICP, LP or negative. To evaluate our DL approach, we divided the images into training (n = 436) and test sets (n = 93). We employed transfer learning with pretrained deep neural networks and conducted fivefold cross-validation to assess model performance. Our dataset's class imbalance was addressed using weighted loss and data augmentation strategies. The models were trained for 50 epochs using Pytorch, achieving an image size of 224 × 224 pixels for both convolutional neural networks (CNNs) and the Swin Transformer.

RESULTS

Our study compared six CNNs and the Swin Transformer for AIBD image classification, with the Swin Transformer achieving the highest average validation accuracy (98.5%). On a separate test set, the best model attained an accuracy of 94.6%, demonstrating 95.3% sensitivity and 97.5% specificity across AIBD classes. Visualization with Grad-CAM (class activation mapping) highlighted the model's reliance on characteristic patterns for accurate classification.

CONCLUSIONS

The study highlighted the accuracy of CNNs in identifying DIF features. This approach aids automated analysis and reporting, offering reproducibility, speed, data handling and cost-efficiency. Integrating DL into skin immunofluorescence promises precise diagnostics and streamlined reporting in this branch of dermatology.

Labelling with dynamics: A data-efficient learning paradigm for medical image segmentation

The success of deep learning on image classification and recognition tasks has led to new applications in diverse contexts, including the field of medical imaging. However, two properties of deep neural networks (DNNs) may limit their future use in medical applications. The first is that DNNs require a large amount of labeled training data, and the second is that the deep learning-based models lack interpretability. In this paper, we propose and investigate a data-efficient framework for the task of general medical image segmentation. We address the two aforementioned challenges by introducing domain knowledge in the form of a strong prior into a deep learning framework. This prior is expressed by a customized dynamical system. We performed experiments on two different datasets, namely JSRT and ISIC2016 (heart and lungs segmentation on chest X-ray images and skin lesion segmentation on dermoscopy images). We have achieved competitive results using the same amount of training data compared to the state-of-the-art methods. More importantly, we demonstrate that our framework is extremely data-efficient, and it can achieve reliable results using extremely limited training data. Furthermore, the proposed method is rotationally invariant and insensitive to initialization.

A Review of Predictive Analytics Models in the Oil and Gas Industries

Enhancing the management and monitoring of oil and gas processes demands the development of precise predictive analytic techniques. Over the past two years, oil and its prediction have advanced significantly using conventional and modern machine learning techniques. Several review articles detail the developments in predictive maintenance and the technical and non-technical aspects of influencing the uptake of big data. The absence of references for machine learning techniques impacts the effective optimization of predictive analytics in the oil and gas sectors. This review paper offers readers thorough information on the latest machine learning methods utilized in this industry's predictive analytical modeling. This review covers different forms of machine learning techniques used in predictive analytical modeling from 2021 to 2023 (91 articles). It provides an overview of the details of the papers that were reviewed, describing the model's categories, the data's temporality, field, and name, the dataset's type, predictive analytics (classification, clustering, or prediction), the models' input and output parameters, the performance metrics, the optimal model, and the model's benefits and drawbacks. In addition, suggestions for future research directions to provide insights into the potential applications of the associated knowledge. This review can serve as a guide to enhance the effectiveness of predictive analytics models in the oil and gas industries.

Deep Learning Models of Multi-Scale Lesion Perception Attention Networks for Diagnosis and Staging of Pneumoconiosis: A Comparative Study with Radiologists

Accurate prediction of pneumoconiosis is essential for individualized early prevention and treatment. However, the different manifestations and high heterogeneity among radiologists make it difficult to diagnose and stage pneumoconiosis accurately. Here, based on DR images collected from two centers, a novel deep learning model, namely Multi-scale Lesion-aware Attention Networks (MLANet), is proposed for diagnosis of pneumoconiosis, staging of pneumoconiosis, and screening of stage I pneumoconiosis. A series of indicators including area under the receiver operating characteristic curve, accuracy, recall, precision, and F1 score were used to comprehensively evaluate the performance of the model. The results show that the MLANet model can effectively improve the consistency and efficiency of pneumoconiosis diagnosis. The accuracy of the MLANet model for pneumoconiosis diagnosis on the internal test set, external validation set, and prospective test set reached 97.87%, 98.03%, and 95.40%, respectively, which was close to the level of qualified radiologists. Moreover, the model can effectively screen stage I pneumoconiosis with an accuracy of 97.16%, a recall of 98.25, a precision of 93.42%, and an F1 score of 95.59%, respectively. The built model performs better than the other four classification models. It is expected to be applied in clinical work to realize the automated diagnosis of pneumoconiosis digital chest radiographs, which is of great significance for individualized early prevention and treatment.

VER-Net: a hybrid transfer learning model for lung cancer detection using CT scan images

BACKGROUND

Lung cancer is the second most common cancer worldwide, with over two million new cases per year. Early identification would allow healthcare practitioners to handle it more effectively. The advancement of computer-aided detection systems significantly impacted clinical analysis and decision-making on human disease. Towards this, machine learning and deep learning techniques are successfully being applied. Due to several advantages, transfer learning has become popular for disease detection based on image data.

METHODS

In this work, we build a novel transfer learning model (VER-Net) by stacking three different transfer learning models to detect lung cancer using lung CT scan images. The model is trained to map the CT scan images with four lung cancer classes. Various measures, such as image preprocessing, data augmentation, and hyperparameter tuning, are taken to improve the efficacy of VER-Net. All the models are trained and evaluated using multiclass classifications chest CT images.

RESULTS

The experimental results confirm that VER-Net outperformed the other eight transfer learning models compared with. VER-Net scored 91%, 92%, 91%, and 91.3% when tested for accuracy, precision, recall, and F1-score, respectively. Compared to the state-of-the-art, VER-Net has better accuracy.

CONCLUSION

VER-Net is not only effectively used for lung cancer detection but may also be useful for other diseases for which CT scan images are available.

Boosting knowledge diversity, accuracy, and stability via tri-enhanced distillation for domain continual medical image segmentation

Domain continual medical image segmentation plays a crucial role in clinical settings. This approach enables segmentation models to continually learn from a sequential data stream across multiple domains. However, it faces the challenge of catastrophic forgetting. Existing methods based on knowledge distillation show potential to address this challenge via a three-stage process: distillation, transfer, and fusion. Yet, each stage presents its unique issues that, collectively, amplify the problem of catastrophic forgetting. To address these issues at each stage, we propose a tri-enhanced distillation framework. (1) Stochastic Knowledge Augmentation reduces redundancy in knowledge, thereby increasing both the diversity and volume of knowledge derived from the old network. (2) Adaptive Knowledge Transfer selectively captures critical information from the old knowledge, facilitating a more accurate knowledge transfer. (3) Global Uncertainty-Guided Fusion introduces a global uncertainty view of the dataset to fuse the old and new knowledge with reduced bias, promoting a more stable knowledge fusion. Our experimental results not only validate the feasibility of our approach, but also demonstrate its superior performance compared to state-of-the-art methods. We suggest that our innovative tri-enhanced distillation framework may establish a robust benchmark for domain continual medical image segmentation.

Deep Transfer Learning Using Real-World Image Features for Medical Image Classification, with a Case Study on Pneumonia X-ray Images

Deep learning has profoundly influenced various domains, particularly medical image analysis. Traditional transfer learning approaches in this field rely on models pretrained on domain-specific medical datasets, which limits their generalizability and accessibility. In this study, we propose a novel framework called real-world feature transfer learning, which utilizes backbone models initially trained on large-scale general-purpose datasets such as ImageNet. We evaluate the effectiveness and robustness of this approach compared to models trained from scratch, focusing on the task of classifying pneumonia in X-ray images. Our experiments, which included converting grayscale images to RGB format, demonstrate that real-world-feature transfer learning consistently outperforms conventional training approaches across various performance metrics. This advancement has the potential to accelerate deep learning applications in medical imaging by leveraging the rich feature representations learned from general-purpose pretrained models. The proposed methodology overcomes the limitations of domain-specific pretrained models, thereby enabling accelerated innovation in medical diagnostics and healthcare. From a mathematical perspective, we formalize the concept of real-world feature transfer learning and provide a rigorous mathematical formulation of the problem. Our experimental results provide empirical evidence supporting the effectiveness of this approach, laying the foundation for further theoretical analysis and exploration. This work contributes to the broader understanding of feature transferability across domains and has significant implications for the development of accurate and efficient models for medical image analysis, even in resource-constrained settings.

GravityNet for end-to-end small lesion detection

This paper introduces a novel one-stage end-to-end detector specifically designed to detect small lesions in medical images. Precise localization of small lesions presents challenges due to their appearance and the diverse contextual backgrounds in which they are found. To address this, our approach introduces a new type of pixel-based anchor that dynamically moves towards the targeted lesion for detection. We refer to this new architecture as GravityNet, and the novel anchors as gravity points since they appear to be "attracted" by the lesions. We conducted experiments on two well-established medical problems involving small lesions to evaluate the performance of the proposed approach: microcalcifications detection in digital mammograms and microaneurysms detection in digital fundus images. Our method demonstrates promising results in effectively detecting small lesions in these medical imaging tasks.

Generalizable disease detection using model ensemble on chest X-ray images

In the realm of healthcare, the demand for swift and precise diagnostic tools has been steadily increasing. This study delves into a comprehensive performance analysis of three pre-trained convolutional neural network (CNN) architectures: ResNet50, DenseNet121, and Inception-ResNet-v2. To ensure the broad applicability of our approach, we curated a large-scale dataset comprising a diverse collection of chest X-ray images, that included both positive and negative cases of COVID-19. The models' performance was evaluated using separate datasets for internal validation (from the same source as the training images) and external validation (from different sources). Our examination uncovered a significant drop in network efficacy, registering a 10.66% reduction for ResNet50, a 36.33% decline for DenseNet121, and a 19.55% decrease for Inception-ResNet-v2 in terms of accuracy. Best results were obtained with DenseNet121 achieving the highest accuracy at 96.71% in internal validation and Inception-ResNet-v2 attaining 76.70% accuracy in external validation. Furthermore, we introduced a model ensemble approach aimed at improving network performance when making inferences on images from diverse sources beyond their training data. The proposed method uses uncertainty-based weighting by calculating the entropy in order to assign appropriate weights to the outputs of each network. Our results showcase the effectiveness of the ensemble method in enhancing accuracy up to 97.38% for internal validation and 81.18% for external validation, while maintaining a balanced ability to detect both positive and negative cases.

Segmentability evaluation of back-scattered SEM images of multiphase materials

Segmentation methods are very useful tools in the Electron Microscopy inspection of materials, enabling the extraction of quantitative results from microscopy images. Back-Scattered Electron (BSE) images carry information of the mean atomic number in the interaction volume and hence can be used to quantify the phase composition in multiphase materials. Since phase composition and proportion affects the material properties and hence its applications, the segmentation accuracy of such images rendered of critical importance for material science. In this work, the notion of segmentability for BSE images is proposed to define the ability of an image to be segmented accurately. This notion can be used to guide the image acquisition process so that segmentability is maximized and segmentation accuracy is ensured. An index is devised to quantify segmentability based on a combination of the modified Fisher Discrimination Ratio and of the second Minkowski functional capturing intensity and spatial aspects of BSE images respectively. The suggested Segmentability Index (SI) is validated in synthetic BSE images which are generated with a novel algorithm allowing the independent control of spatial distribution of phases and their grayscale intensity histograms. Additionally, SI is applied in real-synthetic BSE images, where the real greyscale distributions of Ordinary Portland Cement (OPC) clinker crystallographic phases are used, to demonstrate the ability of SI to indicate the optimum choice of critical image acquisition settings leading to the more accurate segmentation output.

A robust approach for multi-type classification of brain tumor using deep feature fusion

Brain tumors can be classified into many different types based on their shape, texture, and location. Accurate diagnosis of brain tumor types can help doctors to develop appropriate treatment plans to save patients' lives. Therefore, it is very crucial to improve the accuracy of this classification system for brain tumors to assist doctors in their treatment. We propose a deep feature fusion method based on convolutional neural networks to enhance the accuracy and robustness of brain tumor classification while mitigating the risk of over-fitting. Firstly, the extracted features of three pre-trained models including ResNet101, DenseNet121, and EfficientNetB0 are adjusted to ensure that the shape of extracted features for the three models is the same. Secondly, the three models are fine-tuned to extract features from brain tumor images. Thirdly, pairwise summation of the extracted features is carried out to achieve feature fusion. Finally, classification of brain tumors based on fused features is performed. The public datasets including Figshare (Dataset 1) and Kaggle (Dataset 2) are used to verify the reliability of the proposed method. Experimental results demonstrate that the fusion method of ResNet101 and DenseNet121 features achieves the best performance, which achieves classification accuracy of 99.18 and 97.24% in Figshare dataset and Kaggle dataset, respectively.

An Improved Nested U-Net Network for Fluorescence In Situ Hybridization Cell Image Segmentation

Fluorescence in situ hybridization (FISH) is a powerful cytogenetic method used to precisely detect and localize nucleic acid sequences. This technique is proving to be an invaluable tool in medical diagnostics and has made significant contributions to biology and the life sciences. However, the number of cells is large and the nucleic acid sequences are disorganized in the FISH images taken using the microscope. Processing and analyzing images is a time-consuming and laborious task for researchers, as it can easily tire the human eyes and lead to errors in judgment. In recent years, deep learning has made significant progress in the field of medical imaging, especially the successful application of introducing the attention mechanism. The attention mechanism, as a key component of deep learning, improves the understanding and interpretation of medical images by giving different weights to different regions of the image, enabling the model to focus more on important features. To address the challenges in FISH image analysis, we combined medical imaging with deep learning to develop the SEAM-Unet++ automated cell contour segmentation algorithm with integrated attention mechanism. The significant advantage of this algorithm is that it improves the accuracy of cell contours in FISH images. Experiments have demonstrated that by introducing the attention mechanism, our method is able to segment cells that are adherent to each other more efficiently.

EEG decoding for datasets with heterogenous electrode configurations using transfer learning graph neural networks

Objective. Brain-machine interfacing (BMI) has greatly benefited from adopting machine learning methods for feature learning that require extensive data for training, which are often unavailable from a single dataset. Yet, it is difficult to combine data across labs or even data within the same lab collected over the years due to the variation in recording equipment and electrode layouts resulting in shifts in data distribution, changes in data dimensionality, and altered identity of data dimensions. Our objective is to overcome this limitation and learn from many different and diverse datasets across labs with different experimental protocols.Approach. To tackle the domain adaptation problem, we developed a novel machine learning framework combining graph neural networks (GNNs) and transfer learning methodologies for non-invasive motor imagery (MI) EEG decoding, as an example of BMI. Empirically, we focus on the challenges of learning from EEG data with different electrode layouts and varying numbers of electrodes. We utilize three MI EEG databases collected using very different numbers of EEG sensors (from 22 channels to 64) and layouts (from custom layouts to 10-20).Main results. Our model achieved the highest accuracy with lower standard deviations on the testing datasets. This indicates that the GNN-based transfer learning framework can effectively aggregate knowledge from multiple datasets with different electrode layouts, leading to improved generalization in subject-independent MI EEG classification.Significance. The findings of this study have important implications for brain-computer-interface research, as they highlight a promising method for overcoming the limitations posed by non-unified experimental setups. By enabling the integration of diverse datasets with varying electrode layouts, our proposed approach can help advance the development and application of BMI technologies.

Analysis of ultrasonographic images using a deep learning-based model as ancillary diagnostic tool for diagnosing gallbladder polyps

BACKGROUND

Accurately diagnosing gallbladder polyps (GBPs) is important to avoid misdiagnosis and overtreatment.

AIMS

To evaluate the efficacy of a deep learning model and the accuracy of a computer-aided diagnosis by physicians for diagnosing GBPs.

METHODS

This retrospective cohort study was conducted from January 2006 to September 2021, and 3,754 images from 263 patients were analyzed. The outcome of this study was the efficacy of the developed deep learning model in discriminating neoplastic GBPs (NGBPs) from non-NGBPs and to evaluate the accuracy of a computer-aided diagnosis with that made by physicians.

RESULTS

The efficacy of discriminating NGBPs from non- NGBPs using deep learning was 0.944 (accuracy, 0.858; sensitivity, 0.856; specificity, 0.861). The accuracy of an unassisted diagnosis of GBP was 0.634, and that of a computer-aided diagnosis was 0.785 (p<0.001). There were no significant differences in the accuracy of a computer-aided diagnosis between experienced (0.835) and inexperienced (0.772) physicians (p = 0.251). A computer-aided diagnosis significantly assisted inexperienced physicians (0.772 vs. 0.614; p < 0.001) but not experienced physicians.

CONCLUSIONS

Deep learning-based models discriminate NGBPs from non- NGBPs with excellent accuracy. As ancillary diagnostic tools, they may assist inexperienced physicians in improving their diagnostic accuracy.

A generalised deep meta-learning model for automated quality control of cardiovascular magnetic resonance images

BACKGROUND AND OBJECTIVES

Cardiovascular magnetic resonance (CMR) imaging is a powerful modality in functional and anatomical assessment for various cardiovascular diseases. Sufficient image quality is essential to achieve proper diagnosis and treatment. A large number of medical images, the variety of imaging artefacts, and the workload of imaging centres are amongst the factors that reveal the necessity of automatic image quality assessment (IQA). However, automated IQA requires access to bulk annotated datasets for training deep learning (DL) models. Labelling medical images is a tedious, costly and time-consuming process, which creates a fundamental challenge in proposing DL-based methods for medical applications. This study aims to present a new method for CMR IQA when there is limited access to annotated datasets.

METHODS

The proposed generalised deep meta-learning model can evaluate the quality by learning tasks in the prior stage and then fine-tuning the resulting model on a small labelled dataset of the desired tasks. This model was evaluated on the data of over 6,000 subjects from the UK Biobank for five defined tasks, including detecting respiratory motion, cardiac motion, Aliasing and Gibbs ringing artefacts and images without artefacts.

RESULTS

The results of extensive experiments show the superiority of the proposed model. Besides, comparing the model's accuracy with the domain adaptation model indicates a significant difference by using only 64 annotated images related to the desired tasks.

CONCLUSION

The proposed model can identify unknown artefacts in images with acceptable accuracy, which makes it suitable for medical applications and quality assessment of large cohorts. CODE AVAILABILITY: https://github.com/HosseinSimchi/META-IQA-CMRImages.

EFF_D_SVM: a robust multi-type brain tumor classification system

Brain tumors are one of the most threatening diseases to human health. Accurate identification of the type of brain tumor is essential for patients and doctors. An automated brain tumor diagnosis system based on Magnetic Resonance Imaging (MRI) can help doctors to identify the type of tumor and reduce their workload, so it is vital to improve the performance of such systems. Due to the challenge of collecting sufficient data on brain tumors, utilizing pre-trained Convolutional Neural Network (CNN) models for brain tumors classification is a feasible approach. The study proposes a novel brain tumor classification system, called EFF_D_SVM, which is developed on the basic of pre-trained EfficientNetB0 model. Firstly, a new feature extraction module EFF_D was proposed, in which the classification layer of EfficientNetB0 was replaced with two dropout layers and two dense layers. Secondly, the EFF_D model was fine-tuned using Softmax, and then features of brain tumor images were extracted using the fine-tuned EFF_D. Finally, the features were classified using Support Vector Machine (SVM). In order to verify the effectiveness of the proposed brain tumor classification system, a series of comparative experiments were carried out. Moreover, to understand the extracted features of the brain tumor images, Grad-CAM technology was used to visualize the proposed model. Furthermore, cross-validation was conducted to verify the robustness of the proposed model. The evaluation metrics including accuracy, F1-score, recall, and precision were used to evaluate proposed system performance. The experimental results indicate that the proposed model is superior to other state-of-the-art models.

Prompt-Based Tuning of Transformer Models for Multi-Center Medical Image Segmentation of Head and Neck Cancer

Medical image segmentation is a vital healthcare endeavor requiring precise and efficient models for appropriate diagnosis and treatment. Vision transformer (ViT)-based segmentation models have shown great performance in accomplishing this task. However, to build a powerful backbone, the self-attention block of ViT requires large-scale pre-training data. The present method of modifying pre-trained models entails updating all or some of the backbone parameters. This paper proposes a novel fine-tuning strategy for adapting a pretrained transformer-based segmentation model on data from a new medical center. This method introduces a small number of learnable parameters, termed prompts, into the input space (less than 1% of model parameters) while keeping the rest of the model parameters frozen. Extensive studies employing data from new unseen medical centers show that the prompt-based fine-tuning of medical segmentation models provides excellent performance regarding the new-center data with a negligible drop regarding the old centers. Additionally, our strategy delivers great accuracy with minimum re-training on new-center data, significantly decreasing the computational and time costs of fine-tuning pre-trained models. Our source code will be made publicly available.

Deep Learning for Medical Image-Based Cancer Diagnosis

(1) Background: The application of deep learning technology to realize cancer diagnosis based on medical images is one of the research hotspots in the field of artificial intelligence and computer vision. Due to the rapid development of deep learning methods, cancer diagnosis requires very high accuracy and timeliness as well as the inherent particularity and complexity of medical imaging. A comprehensive review of relevant studies is necessary to help readers better understand the current research status and ideas. (2) Methods: Five radiological images, including X-ray, ultrasound (US), computed tomography (CT), magnetic resonance imaging (MRI), positron emission computed tomography (PET), and histopathological images, are reviewed in this paper. The basic architecture of deep learning and classical pretrained models are comprehensively reviewed. In particular, advanced neural networks emerging in recent years, including transfer learning, ensemble learning (EL), graph neural network, and vision transformer (ViT), are introduced. Five overfitting prevention methods are summarized: batch normalization, dropout, weight initialization, and data augmentation. The application of deep learning technology in medical image-based cancer analysis is sorted out. (3) Results: Deep learning has achieved great success in medical image-based cancer diagnosis, showing good results in image classification, image reconstruction, image detection, image segmentation, image registration, and image synthesis. However, the lack of high-quality labeled datasets limits the role of deep learning and faces challenges in rare cancer diagnosis, multi-modal image fusion, model explainability, and generalization. (4) Conclusions: There is a need for more public standard databases for cancer. The pre-training model based on deep neural networks has the potential to be improved, and special attention should be paid to the research of multimodal data fusion and supervised paradigm. Technologies such as ViT, ensemble learning, and few-shot learning will bring surprises to cancer diagnosis based on medical images.

Attention-Guided Transfer Learning for Identification of Filamentous Fungi Encountered in the Clinical Laboratory

This study addresses the challenge of accurately identifying filamentous fungi in medical laboratories using transfer learning with convolutional neural networks (CNNs). The study uses microscopic images from touch-tape slides with lactophenol cotton blue staining, the most common method in clinical settings, to classify fungal genera and identify Aspergillus species. The training and test data sets included 4,108 images with representative microscopic morphology for each genus, and a soft attention mechanism was incorporated to enhance classification accuracy. As a result, the study achieved an overall classification accuracy of 94.9% for four frequently encountered genera and 84.5% for Aspergillus species. One of the distinct features is the involvement of medical technologists in developing a model that seamlessly integrates into routine workflows. In addition, the study highlights the potential of merging advanced technology with medical laboratory practices to diagnose filamentous fungi accurately and efficiently. IMPORTANCE This study utilizes transfer learning with CNNs to classify fungal genera and identify Aspergillus species using microscopic images from touch-tape preparation and lactophenol cotton blue staining. The training and test data sets included 4,108 images with representative microscopic morphology for each genus, and a soft attention mechanism was incorporated to enhance classification accuracy. As a result, the study achieved an overall classification accuracy of 94.9% for four frequently encountered genera and 84.5% for Aspergillus species. One of the distinct features is the involvement of medical technologists in developing a model that seamlessly integrates into routine workflows. In addition, the study highlights the potential of merging advanced technology with medical laboratory practices to diagnose filamentous fungi accurately and efficiently.

Developing a Supplementary Diagnostic Tool for Breast Cancer Risk Estimation Using Ensemble Transfer Learning

Breast cancer is the most prevalent cancer worldwide. Thus, it is necessary to improve the efficiency of the medical workflow of the disease. Therefore, this study aims to develop a supplementary diagnostic tool for radiologists using ensemble transfer learning and digital mammograms. The digital mammograms and their associated information were collected from the department of radiology and pathology at Hospital Universiti Sains Malaysia. Thirteen pre-trained networks were selected and tested in this study. ResNet101V2 and ResNet152 had the highest mean PR-AUC, MobileNetV3Small and ResNet152 had the highest mean precision, ResNet101 had the highest mean F1 score, and ResNet152 and ResNet152V2 had the highest mean Youden J index. Subsequently, three ensemble models were developed using the top three pre-trained networks whose ranking was based on PR-AUC values, precision, and F1 scores. The final ensemble model, which consisted of Resnet101, Resnet152, and ResNet50V2, had a mean precision value, F1 score, and Youden J index of 0.82, 0.68, and 0.12, respectively. Additionally, the final model demonstrated balanced performance across mammographic density. In conclusion, this study demonstrates the good performance of ensemble transfer learning and digital mammograms in breast cancer risk estimation. This model can be utilised as a supplementary diagnostic tool for radiologists, thus reducing their workloads and further improving the medical workflow in the screening and diagnosis of breast cancer.

Machine Learning for Brain MRI Data Harmonisation: A Systematic Review

BACKGROUND

Magnetic Resonance Imaging (MRI) data collected from multiple centres can be heterogeneous due to factors such as the scanner used and the site location. To reduce this heterogeneity, the data needs to be harmonised. In recent years, machine learning (ML) has been used to solve different types of problems related to MRI data, showing great promise.

OBJECTIVE

This study explores how well various ML algorithms perform in harmonising MRI data, both implicitly and explicitly, by summarising the findings in relevant peer-reviewed articles. Furthermore, it provides guidelines for the use of current methods and identifies potential future research directions.

METHOD

This review covers articles published through PubMed, Web of Science, and IEEE databases through June 2022. Data from studies were analysed based on the criteria of Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). Quality assessment questions were derived to assess the quality of the included publications.

RESULTS

a total of 41 articles published between 2015 and 2022 were identified and analysed. In the review, MRI data has been found to be harmonised either in an implicit (n = 21) or an explicit (n = 20) way. Three MRI modalities were identified: structural MRI (n = 28), diffusion MRI (n = 7) and functional MRI (n = 6).

CONCLUSION

Various ML techniques have been employed to harmonise different types of MRI data. There is currently a lack of consistent evaluation methods and metrics used across studies, and it is recommended that the issue be addressed in future studies. Harmonisation of MRI data using ML shows promises in improving performance for ML downstream tasks, while caution should be exercised when using ML-harmonised data for direct interpretation.

Convolutional Neural Networks to Classify Alzheimer’s Disease Severity Based on SPECT Images: A Comparative Study

Image recognition and neuroimaging are increasingly being used to understand the progression of Alzheimer’s disease (AD). However, image data from single-photon emission computed tomography (SPECT) are limited. Medical image analysis requires large, labeled training datasets. Therefore, studies have focused on overcoming this problem. In this study, the detection performance of five convolutional neural network (CNN) models (MobileNet V2 and NASNetMobile (lightweight models); VGG16, Inception V3, and ResNet (heavier weight models)) on medical images was compared to establish a classification model for epidemiological research. Brain scan image data were collected from 99 subjects, and 4711 images were used. Demographic data were compared using the chi-squared test and one-way analysis of variance with Bonferroni’s post hoc test. Accuracy and loss functions were used to evaluate the performance of CNN models. The cognitive abilities screening instrument and mini mental state exam scores of subjects with a clinical dementia rating (CDR) of 2 were considerably lower than those of subjects with a CDR of 1 or 0.5. This study analyzed the classification performance of various CNN models for medical images and proved the effectiveness of transfer learning in identifying the mild cognitive impairment, mild AD, and moderate AD scoring based on SPECT images.

Robust landmark-based brain shift correction with a Siamese neural network in ultrasound-guided brain tumor resection

PURPOSE

In brain tumor surgery, tissue shift (called brain shift) can move the surgical target and invalidate the surgical plan. A cost-effective and flexible tool, intra-operative ultrasound (iUS) with robust image registration algorithms can effectively track brain shift to ensure surgical outcomes and safety.

METHODS

We proposed to employ a Siamese neural network, which was first trained using natural images and fine-tuned with domain-specific data to automatically detect matching anatomical landmarks in iUS scans at different surgical stages. An efficient 2.5D approach and an iterative re-weighted least squares algorithm are utilized to perform landmark-based registration for brain shift correction. The proposed method is validated and compared against the state-of-the-art methods using the public BITE and RESECT datasets.

RESULTS

Registration of pre-resection iUS scans to during- and post-resection iUS images were executed. The results with the proposed method shows a significant improvement from the initial misalignment ([Formula: see text]) and the method is comparable to the state-of-the-art methods validated on the same datasets.

CONCLUSIONS

We have proposed a robust technique to efficiently detect matching landmarks in iUS and perform brain shift correction with excellent performance. It has the potential to improve the accuracy and safety of neurosurgery.

Synchronous Medical Image Augmentation framework for deep learning-based image segmentation

Various deep learning (DL) models are widely applied in medical image analysis, and their performance depends on the scale and diversity of available training data. However, medical images often suffer from difficulty in data acquisition, imbalance in sample categories, and high cost of labeling. In addition, most image augmentation approaches mainly focus on image synthesis only for classification tasks, and rarely consider the synthetic image-label pairs for image segmentation tasks. In this paper, we focus on the medical image augmentation for DL-based image segmentation and the synchronization between augmented image samples and their labels. We design a Synchronous Medical Image Augmentation (SMIA) framework, which includes two modules based on stochastic transformation and synthesis, and provides diverse and annotated training sets for DL models. In the transform-based SMIA module, for each medical image sample and its tissue segments, a subset of SMIA factors with a random number of factors and stochastic parameter values are selected to simultaneously generate augmented samples and the paired tissue segments. In the synthesis-based SMIA module, we randomly replace the original tissues with the augmented tissues using an equivalent replacement method to synthesize new medical images, which can well maintain the original medical implications. DL-based image segmentation experiments on bone marrow smear and dermoscopic images demonstrate that the proposed SMIA framework can generate category-balanced and diverse training data, and have a positive impact on the performance of the models.

A multi-object deep neural network architecture to detect prostate anatomy in T2-weighted MRI: Performance evaluation

Prostate gland segmentation is the primary step to estimate gland volume, which aids in the prostate disease management. In this study, we present a 2D-3D convolutional neural network (CNN) ensemble that automatically segments the whole prostate gland along with the peripheral zone (PZ) (PPZ-SegNet) using a T2-weighted sequence (T2W) of Magnetic Resonance Imaging (MRI). The study used 4 different public data sets organized as Train #1 and Test #1 (independently derived from the same cohort), Test #2, Test #3 and Test #4. The prostate gland and the peripheral zone (PZ) anatomy were manually delineated with consensus read by a radiologist, except for Test #4 cohorts that had pre-marked glandular anatomy. A Bayesian hyperparameter optimization method was applied to construct the network model (PPZ-SegNet) with a training cohort (Train #1, n = 150) using a five-fold cross validation. The model evaluation was performed on an independent cohort of 283 T2W MRI prostate cases (Test #1 to #4) without any additional tuning. The data cohorts were derived from The Cancer Imaging Archives (TCIA): PROSTATEx Challenge, Prostatectomy, Repeatability studies and PROMISE12-Challenge. The segmentation performance was evaluated by computing the Dice similarity coefficient and Hausdorff distance between the estimated-deep-network identified regions and the radiologist-drawn annotations. The deep network architecture was able to segment the prostate gland anatomy with an average Dice score of 0.86 in Test #1 (n = 192), 0.79 in Test #2 (n = 26), 0.81 in Test #3 (n = 15), and 0.62 in Test #4 (n = 50). We also found the Dice coefficient improved with larger prostate volumes in 3 of the 4 test cohorts. The variation of the Dice scores from different cohorts of test images suggests the necessity of more diverse models that are inclusive of dependencies such as the gland sizes and others, which will enable us to develop a universal network for prostate and PZ segmentation. Our training and evaluation code can be accessed through the link: https://github.com/mariabaldeon/PPZ-SegNet.git.

Recognizing pathology of renal tumor from macroscopic cross-section image by deep learning

OBJECTIVES

This study aims to develop and evaluate the deep learning-based classification model for recognizing the pathology of renal tumor from macroscopic cross-section image.

METHODS

A total of 467 pathology-confirmed patients who received radical nephrectomy or partial nephrectomy were retrospectively enrolled. The experiment of distinguishing malignant and benign renal tumor are conducted followed by performing the multi-subtypes classification models for recognizing four subtypes of benign tumor and four subtypes of malignant tumors, respectively. The classification models used the same backbone networks which are based on the convolutional neural network (CNN), including EfficientNet-B4, ResNet-18, and VGG-16. The performance of the classification models was evaluated by area under the receiver operating characteristic curve (AUC), sensitivity, specificity, and accuracy. Besides, we performed the quantitative comparison among these CNN models.

RESULTS

For the model to differentiate the malignant tumor from the benign tumor, three CNN models all obtained relatively satisfactory performance and the highest AUC was achieved by the ResNet-18 model (AUC = 0.9226). There is not statistically significance between EfficientNet-B4 and ResNet-18 architectures and both of them are significantly statistically better than the VGG-16 model. The micro-averaged AUC, macro-averaged sensitivity, macro-averaged specificity, and micro-averaged accuracy for the VGG-16 model to distinguish the malignant tumor subtypes achieved 0.9398, 0.5774, 0.8660, and 0.7917, respectively. The performance of the EfficientNet-B4 is not better than that of VGG-16 in terms of micro-averaged AUC except for other metrics. For the models to recognize the benign tumor subtypes, the EfficientNet-B4 ranked the best performance, but had no significantly statistical difference with other two models with respect to micro-averaged AUC.

CONCLUSIONS

The classification results were relatively satisfactory, which showed the potential for clinical application when analyzing the renal tumor macroscopic cross-section images. Automatically distinguishing the malignant tumor from benign tumor and identifying the subtypes pathology of renal tumor could make the patient-management process more efficient.

Light-Dermo: A Lightweight Pretrained Convolution Neural Network for the Diagnosis of Multiclass Skin Lesions

Skin cancer develops due to the unusual growth of skin cells. Early detection is critical for the recognition of multiclass pigmented skin lesions (PSLs). At an early stage, the manual work by ophthalmologists takes time to recognize the PSLs. Therefore, several "computer-aided diagnosis (CAD)" systems are developed by using image processing, machine learning (ML), and deep learning (DL) techniques. Deep-CNN models outperformed traditional ML approaches in extracting complex features from PSLs. In this study, a special transfer learning (TL)-based CNN model is suggested for the diagnosis of seven classes of PSLs. A novel approach (Light-Dermo) is developed that is based on a lightweight CNN model and applies the channelwise attention (CA) mechanism with a focus on computational efficiency. The ShuffleNet architecture is chosen as the backbone, and squeeze-and-excitation (SE) blocks are incorporated as the technique to enhance the original ShuffleNet architecture. Initially, an accessible dataset with 14,000 images of PSLs from seven classes is used to validate the Light-Dermo model. To increase the size of the dataset and control its imbalance, we have applied data augmentation techniques to seven classes of PSLs. By applying this technique, we collected 28,000 images from the HAM10000, ISIS-2019, and ISIC-2020 datasets. The outcomes of the experiments show that the suggested approach outperforms compared techniques in many cases. The most accurately trained model has an accuracy of 99.14%, a specificity of 98.20%, a sensitivity of 97.45%, and an F1-score of 98.1%, with fewer parameters compared to state-of-the-art DL models. The experimental results show that Light-Dermo assists the dermatologist in the better diagnosis of PSLs. The Light-Dermo code is available to the public on GitHub so that researchers can use it and improve it.

NSCGCN: A novel deep GCN model to diagnosis COVID-19

AIM

Corona Virus Disease 2019 (COVID-19) was a lung disease with high mortality and was highly contagious. Early diagnosis of COVID-19 and distinguishing it from pneumonia was beneficial for subsequent treatment.

OBJECTIVES

Recently, Graph Convolutional Network (GCN) has driven a significant contribution to disease diagnosis. However, limited by the nature of the graph convolution algorithm, deep GCN has an over-smoothing problem. Most of the current GCN models are shallow neural networks, which do not exceed five layers. Furthermore, the objective of this study is to develop a novel deep GCN model based on the DenseGCN and the pre-trained model of deep Convolutional Neural Network (CNN) to complete the diagnosis of chest X-ray (CXR) images.

METHODS

We apply the pre-trained model of deep CNN to perform feature extraction on the data to complete the extraction of pixel-level features in the image. And then, to extract the potential relationship between the obtained features, we propose Neighbourhood Feature Reconstruction Algorithm to reconstruct them into graph-structured data. Finally, we design a deep GCN model that exploits the graph-structured data to diagnose COVID-19 effectively. In the deep GCN model, we propose a Node-Self Convolution Algorithm (NSC) based on feature fusion to construct a deep GCN model called NSCGCN (Node-Self Convolution Graph Convolutional Network).

RESULTS

Experiments were carried out on the Computed Tomography (CT) and CXR datasets. The results on the CT dataset confirmed that: compared with the six state-of-the-art (SOTA) shallow GCN models, the accuracy and sensitivity of the proposed NSCGCN had improve 8% as sensitivity (Sen.) = 87.50%, F1 score = 97.37%, precision (Pre.) = 89.10%, accuracy (Acc.) = 97.50%, area under the ROC curve (AUC) = 97.09%. Moreover, the results on the CXR dataset confirmed that: compared with the fourteen SOTA GCN models, sixteen SOTA CNN transfer learning models and eight SOTA COVID-19 diagnosis methods on the COVID-19 dataset. Our proposed method had best performances as Sen. = 96.45%, F1 score = 96.45%, Pre. = 96.61%, Acc. = 96.45%, AUC = 99.22%.

CONCLUSION

Our proposed NSCGCN model is effective and performed better than the thirty-eight SOTA methods. Thus, the proposed NSC could help build deep GCN models. Our proposed COVID-19 diagnosis method based on the NSCGCN model could help radiologists detect pneumonia from CXR images and distinguish COVID-19 from Ordinary Pneumonia (OPN). The source code of this work will be publicly available at https://github.com/TangChaosheng/NSCGCN.

A Transfer Learning Approach with a Convolutional Neural Network for the Classification of Lung Carcinoma

Lung cancer is among the most hazardous types of cancer in humans. The correct diagnosis of pathogenic lung disease is critical for medication. Traditionally, determining the pathological form of lung cancer involves an expensive and time-consuming process investigation. Lung cancer is a leading cause of mortality worldwide, with lung tissue nodules being the most prevalent way for doctors to identify it. The proposed model is based on robust deep-learning-based lung cancer detection and recognition. This study uses a deep neural network as an extraction of features approach in a computer-aided diagnosing (CAD) system to assist in detecting lung illnesses at high definition. The proposed model is categorized into three phases: first, data augmentation is performed, classification is then performed using the pretrained CNN model, and lastly, localization is completed. The amount of obtained data in medical image assessment is occasionally inadequate to train the learning network. We train the classifier using a technique known as transfer learning (TL) to solve the issue introduced into the process. The proposed methodology offers a non-invasive diagnostic tool for use in the clinical assessment that is effective. The proposed model has a lower number of parameters that are much smaller compared to the state-of-the-art models. We also examined the desired dataset’s robustness depending on its size. The standard performance metrics are used to assess the effectiveness of the proposed architecture. In this dataset, all TL techniques perform well, and VGG 16, VGG 19, and Xception for 20 epoch structure are compared. Preprocessing functions as a wonderful bridge to build a dependable model and eventually helps to forecast future scenarios by including the interface at a faster phase for any model. At the 20th epoch, the accuracy of VGG 16, VGG 19, and Xception is 98.83 percent, 98.05 percent, and 97.4 percent.

Design and Implementation of Chinese Common Braille Translation System Integrating Braille Word Segmentation and Concatenation Rules

An important sign of the accessibility of Braille information is the realization of the mutual translation between Chinese and the Braille. Due to the irregularity and uncertainty of the Prevailing Mandarin Braille, coupled with the lack of a large-scale Braille corpus, the quality of Chinese-Braille translation seems to be poor. In July 2018, the National Language Commission released the “Chinese Common Braille Scheme” and advocated replacing the “Prevailing Mandarin Braille.” Aimed at improving translation accuracy, this research, which is based on the self-built Chinese Common Braille corpus and combined with the HanLP (Han Language Processing) dictionary and the Chinese-Braille word corpus (a Braille word segmentation and concatenation dictionary for generating a unigram language model), uses the n-gram language model to design and implement a Chinese-Braille intertranslation system that integrates Chinese and Braille Word Segmentation and Concatenation Rules. More importantly, this research proposes an experimental plan for improving the Braille Word Segmentation and Concatenation Rules using a Chinese-Braille word corpus. Experiments show that in the field of educational literature, the accuracy rate of translation from Chinese to Chinese Common Braille has reached 95.01%, and the accuracy of Chinese Common Braille to Chinese translation has reached 90.15%.