A survey of label-noise deep learning for medical image analysis

Several factors are associated with the success of deep learning. One of the most important reasons is the availability of large-scale datasets with clean annotations. However, obtaining datasets with accurate labels in the medical imaging domain is challenging. The reliability and consistency of medical labeling are some of these issues, and low-quality annotations with label noise usually exist. Because noisy labels reduce the generalization performance of deep neural networks, learning with noisy labels is becoming an essential task in medical image analysis. Literature on this topic has expanded in terms of volume and scope. However, no recent surveys have collected and organized this knowledge, impeding the ability of researchers and practitioners to utilize it. In this work, we presented an up-to-date survey of label-noise learning for medical image domain. We reviewed extensive literature, illustrated some typical methods, and showed unified taxonomies in terms of methodological differences. Subsequently, we conducted the methodological comparison and demonstrated the corresponding advantages and disadvantages. Finally, we discussed new research directions based on the characteristics of medical images. Our survey aims to provide researchers and practitioners with a solid understanding of existing medical label-noise learning, such as the main algorithms developed over the past few years, which could help them investigate new methods to combat with the negative effects of label noise.

Reviewing 3D convolutional neural network approaches for medical image segmentation

Background

Convolutional neural networks (CNNs) assume pivotal roles in aiding clinicians in diagnosis and treatment decisions. The rapid evolution of imaging technology has established three-dimensional (3D) CNNs as a formidable framework for delineating organs and anomalies in medical images. The prominence of 3D CNN frameworks is steadily growing within medical image segmentation and classification. Thus, our proposition entails a comprehensive review, encapsulating diverse 3D CNN algorithms for the segmentation of medical image anomalies and organs.

Methods

This study systematically presents an exhaustive review of recent 3D CNN methodologies. Rigorous screening of abstracts and titles were carried out to establish their relevance. Research papers disseminated across academic repositories were meticulously chosen, analyzed, and appraised against specific criteria. Insights into the realm of anomalies and organ segmentation were derived, encompassing details such as network architecture and achieved accuracies.

Results

This paper offers an all-encompassing analysis, unveiling the prevailing trends in 3D CNN segmentation. In-depth elucidations encompass essential insights, constraints, observations, and avenues for future exploration. A discerning examination indicates the preponderance of the encoder-decoder network in segmentation tasks. The encoder-decoder framework affords a coherent methodology for the segmentation of medical images.

Conclusion

The findings of this study are poised to find application in clinical diagnosis and therapeutic interventions. Despite inherent limitations, CNN algorithms showcase commendable accuracy levels, solidifying their potential in medical image segmentation and classification endeavors.

Deep local-to-global feature learning for medical image super-resolution

Medical images play a vital role in medical analysis by providing crucial information about patients' pathological conditions. However, the quality of these images can be compromised by many factors, such as limited resolution of the instruments, artifacts caused by movements, and the complexity of the scanned areas. As a result, low-resolution (LR) images cannot provide sufficient information for diagnosis. To address this issue, researchers have attempted to apply image super-resolution (SR) techniques to restore the high-resolution (HR) images from their LR counterparts. However, these techniques are designed for generic images, and thus suffer from many challenges unique to medical images. An obvious one is the diversity of the scanned objects; for example, the organs, tissues, and vessels typically appear in different sizes and shapes, and are thus hard to restore with standard convolution neural networks (CNNs). In this paper, we develop a dynamic-local learning framework to capture the details of these diverse areas, consisting of deformable convolutions with adjustable kernel shapes. Moreover, the global information between the tissues and organs is vital for medical diagnosis. To preserve global information, we propose pixel-pixel and patch-patch global learning using a non-local mechanism and a vision transformer (ViT), respectively. The result is a novel CNN-ViT neural network with Local-to-Global feature learning for medical image SR, referred to as LGSR, which can accurately restore both local details and global information. We evaluate our method on six public datasets and one large-scale private dataset, which include five different types of medical images (i.e., Ultrasound, OCT, Endoscope, CT, and MRI images). Experiments show that the proposed method achieves superior PSNR/SSIM and visual performance than the state of the arts with competitive computational costs, measured in network parameters, runtime, and FLOPs. What is more, the experiment conducted on OCT image segmentation for the downstream task demonstrates a significantly positive performance effect of LGSR.

Evaluation metrics and statistical tests for machine learning

Research on different machine learning (ML) has become incredibly popular during the past few decades. However, for some researchers not familiar with statistics, it might be difficult to understand how to evaluate the performance of ML models and compare them with each other. Here, we introduce the most common evaluation metrics used for the typical supervised ML tasks including binary, multi-class, and multi-label classification, regression, image segmentation, object detection, and information retrieval. We explain how to choose a suitable statistical test for comparing models, how to obtain enough values of the metric for testing, and how to perform the test and interpret its results. We also present a few practical examples about comparing convolutional neural networks used to classify X-rays with different lung infections and detect cancer tumors in positron emission tomography images.

HPFG: semi-supervised medical image segmentation framework based on hybrid pseudo-label and feature-guiding

Semi-supervised learning methods have been attracting much attention in medical image segmentation due to the lack of high-quality annotation. To cope with the noise problem of pseudo-label in semi-supervised medical image segmentation and the limitations of contrastive learning applications, we propose a semi-supervised medical image segmentation framework, HPFG, based on hybrid pseudo-label and feature-guiding, which consists of a hybrid pseudo-label strategy and two different feature-guiding modules. The hybrid pseudo-label strategy uses the CutMix operation and an auxiliary network to enable the labeled images to guide the unlabeled images to generate high-quality pseudo-label and reduce the impact of pseudo-label noise. In addition, a feature-guiding encoder module based on feature-level contrastive learning is designed to guide the encoder to mine useful local and global image features, thus effectively enhancing the feature extraction capability of the model. At the same time, a feature-guiding decoder module based on adaptive class-level contrastive learning is designed to guide the decoder in better extracting class information, achieving intra-class affinity and inter-class separation, and effectively alleviating the class imbalance problem in medical datasets. Extensive experimental results show that the segmentation performance of the HPFG framework proposed in this paper outperforms existing semi-supervised medical image segmentation methods on three public datasets: ACDC, LIDC, and ISIC. Code is available at https://github.com/fakerlove1/HPFG .

Robust Medical Diagnosis: A Novel Two-Phase Deep Learning Framework for Adversarial Proof Disease Detection in Radiology Images

In the realm of medical diagnostics, the utilization of deep learning techniques, notably in the context of radiology images, has emerged as a transformative force. The significance of artificial intelligence (AI), specifically machine learning (ML) and deep learning (DL), lies in their capacity to rapidly and accurately diagnose diseases from radiology images. This capability has been particularly vital during the COVID-19 pandemic, where rapid and precise diagnosis played a pivotal role in managing the spread of the virus. DL models, trained on vast datasets of radiology images, have showcased remarkable proficiency in distinguishing between normal and COVID-19-affected cases, offering a ray of hope amidst the crisis. However, as with any technological advancement, vulnerabilities emerge. Deep learning-based diagnostic models, although proficient, are not immune to adversarial attacks. These attacks, characterized by carefully crafted perturbations to input data, can potentially disrupt the models' decision-making processes. In the medical context, such vulnerabilities could have dire consequences, leading to misdiagnoses and compromised patient care. To address this, we propose a two-phase defense framework that combines advanced adversarial learning and adversarial image filtering techniques. We use a modified adversarial learning algorithm to enhance the model's resilience against adversarial examples during the training phase. During the inference phase, we apply JPEG compression to mitigate perturbations that cause misclassification. We evaluate our approach on three models based on ResNet-50, VGG-16, and Inception-V3. These models perform exceptionally in classifying radiology images (X-ray and CT) of lung regions into normal, pneumonia, and COVID-19 pneumonia categories. We then assess the vulnerability of these models to three targeted adversarial attacks: fast gradient sign method (FGSM), projected gradient descent (PGD), and basic iterative method (BIM). The results show a significant drop in model performance after the attacks. However, our defense framework greatly improves the models' resistance to adversarial attacks, maintaining high accuracy on adversarial examples. Importantly, our framework ensures the reliability of the models in diagnosing COVID-19 from clean images.

Multi-Scale and Spatial Information Extraction for Kidney Tumor Segmentation: A Contextual Deformable Attention and Edge-Enhanced U-Net

Kidney tumor segmentation is a difficult task because of the complex spatial and volumetric information present in medical images. Recent advances in deep convolutional neural networks (DCNNs) have improved tumor segmentation accuracy. However, the practical usability of current CNN-based networks is constrained by their high computational complexity. Additionally, these techniques often struggle to make adaptive modifications based on the structure of the tumors, which can lead to blurred edges in segmentation results. A lightweight architecture called the contextual deformable attention and edge-enhanced U-Net (CDA2E-Net) for high-accuracy pixel-level kidney tumor segmentation is proposed to address these challenges. Rather than using complex deep encoders, the approach includes a lightweight depthwise dilated ShuffleNetV2 (LDS-Net) encoder integrated into the CDA2E-Net framework. The proposed method also contains a multiscale attention feature pyramid pooling (MAF2P) module that improves the ability of multiscale features to adapt to various tumor shapes. Finally, an edge-enhanced loss function is introduced to guide the CDA2E-Net to concentrate on tumor edge information. The CDA2E-Net is evaluated on the KiTS19 and KiTS21 datasets, and the results demonstrate its superiority over existing approaches in terms of Hausdorff distance (HD), intersection over union (IoU), and dice coefficient (DSC) metrics.

Reduction of overfitting on the highly imbalanced ISIC-2019 skin dataset using deep learning frameworks

BACKGROUND

With the rapid growth of Deep Neural Networks (DNN) and Computer-Aided Diagnosis (CAD), more significant works have been analysed for cancer related diseases. Skin cancer is the most hazardous type of cancer that cannot be diagnosed in the early stages.

OBJECTIVE

The diagnosis of skin cancer is becoming a challenge to dermatologists as an abnormal lesion looks like an ordinary nevus at the initial stages. Therefore, early identification of lesions (origin of skin cancer) is essential and helpful for treating skin cancer patients effectively. The enormous development of automated skin cancer diagnosis systems significantly supports dermatologists.

METHODS

This paper performs a classification of skin cancer by utilising various deep-learning frameworks after resolving the class Imbalance problem in the ISIC-2019 dataset. A fine-tuned ResNet-50 model is used to evaluate the performance of original data, augmented data, and after by adding the focal loss. Focal loss is the best technique to solve overfitting problems by assigning weights to hard misclassified images.

RESULTS

Finally, augmented data with focal loss is given a good classification performance with 98.85% accuracy, 95.52% precision, and 95.93% recall. Matthews Correlation coefficient (MCC) is the best metric to evaluate the quality of multi-class images. It has given outstanding performance by using augmented data and focal loss.

HCS-Net: Multi-level deformation strategy combined with quadruple attention for image registration

BACKGROUND AND OBJECTIVE

Non-rigid image registration plays a significant role in computer-aided diagnosis and surgical navigation for brain diseases. Registration methods that utilize convolutional neural networks (CNNs) have shown excellent accuracy when applied to brain magnetic resonance images (MRI). However, CNNs have limitations in understanding long-range spatial relationships in images, which makes it challenging to incorporate contextual information. And in intricate image registration tasks, it is difficult to achieve a satisfactory dense prediction field, resulting in poor registration performance.

METHODS

This paper proposes a multi-level deformable unsupervised registration model that combines Transformer and CNN to achieve non-rigid registration of brain MRI. Firstly, utilizing a dual encoder structure to establish the dependency relationship between the global features of two images and to merge features of varying scales, as well as to preserve the relative spatial position information of feature maps at different scales. Then the proposed multi-level deformation strategy utilizes different deformable fields of varying resolutions generated by the decoding structure to progressively deform the moving image. Ultimately, the proposed quadruple attention module is incorporated into the decoding structure to merge feature information from various directions and emphasize the spatial features in the dominant channels.

RESULTS

The experimental results on multiple brain MR datasets demonstrate that the promising network could provide accurate registration and is comparable to state-of-the-art methods.

CONCLUSION

The proposed registration model can generate superior deformable fields and achieve more precise registration effects, enhancing the auxiliary role of medical image registration in various fields and advancing the development of computer-aided diagnosis, surgical navigation, and related domains.

A scoping review on multimodal deep learning in biomedical images and texts

OBJECTIVE

Computer-assisted diagnostic and prognostic systems of the future should be capable of simultaneously processing multimodal data. Multimodal deep learning (MDL), which involves the integration of multiple sources of data, such as images and text, has the potential to revolutionize the analysis and interpretation of biomedical data. However, it only caught researchers' attention recently. To this end, there is a critical need to conduct a systematic review on this topic, identify the limitations of current work, and explore future directions.

METHODS

In this scoping review, we aim to provide a comprehensive overview of the current state of the field and identify key concepts, types of studies, and research gaps with a focus on biomedical images and texts joint learning, mainly because these two were the most commonly available data types in MDL research.

RESULT

This study reviewed the current uses of multimodal deep learning on five tasks: (1) Report generation, (2) Visual question answering, (3) Cross-modal retrieval, (4) Computer-aided diagnosis, and (5) Semantic segmentation.

CONCLUSION

Our results highlight the diverse applications and potential of MDL and suggest directions for future research in the field. We hope our review will facilitate the collaboration of natural language processing (NLP) and medical imaging communities and support the next generation of decision-making and computer-assisted diagnostic system development.

Cross-domain mechanism for few-shot object detection on Urine Sediment Image

Deep learning object detection networks require a large amount of box annotation data for training, which is difficult to obtain in the medical image field. The few-shot object detection algorithm is significant for an unseen category, which can be identified and localized with a few labeled data. For medical image datasets, the image style and target features are incredibly different from the knowledge obtained from training on the original dataset. We propose a background suppression attention(BSA) and feature space fine-tuning module (FSF) for this cross-domain situation where there is a large gap between the source and target domains. The background suppression attention reduces the influence of background information in the training process. The feature space fine-tuning module adjusts the feature distribution of the interest features, which helps to make better predictions. Our approach improves detection performance by using only the information extracted from the model without maintaining additional information, which is convenient and can be easily plugged into other networks. We evaluate the detection performance in the in-domain situation and cross-domain situation. In-domain experiments on the VOC and COCO datasets and the cross-domain experiments on the VOC to medical image dataset UriSed2K show that our proposed method effectively improves the few-shot detection performance.

AugmentA: Patient-specific augmented atrial model generation tool

Digital twins of patients' hearts are a promising tool to assess arrhythmia vulnerability and to personalize therapy. However, the process of building personalized computational models can be challenging and requires a high level of human interaction. We propose a patient-specific Augmented Atria generation pipeline (AugmentA) as a highly automated framework which, starting from clinical geometrical data, provides ready-to-use atrial personalized computational models. AugmentA identifies and labels atrial orifices using only one reference point per atrium. If the user chooses to fit a statistical shape model to the input geometry, it is first rigidly aligned with the given mean shape before a non-rigid fitting procedure is applied. AugmentA automatically generates the fiber orientation and finds local conduction velocities by minimizing the error between the simulated and clinical local activation time (LAT) map. The pipeline was tested on a cohort of 29 patients on both segmented magnetic resonance images (MRI) and electroanatomical maps of the left atrium. Moreover, the pipeline was applied to a bi-atrial volumetric mesh derived from MRI. The pipeline robustly integrated fiber orientation and anatomical region annotations in 38.4 ± 5.7 s. In conclusion, AugmentA offers an automated and comprehensive pipeline delivering atrial digital twins from clinical data in procedural time.

Radiomics under 2D regions, 3D regions, and peritumoral regions reveal tumor heterogeneity in non-small cell lung cancer: a multicenter study

PURPOSE

Lung cancer has significant genetic and phenotypic heterogeneity, leading to poor prognosis. Radiomic features have emerged as promising predictors of the tumor phenotype. However, the role of underlying information surrounding the cancer remains unclear.

MATERIALS AND METHODS

We conducted a retrospective study of 508 patients with NSCLC from three institutions. Radiomics models were built using features from six tumor regions and seven classifiers to predict three prognostically significant tumor phenotypes. The models were evaluated and interpreted by the mean area under the receiver operating characteristic curve (AUC) under nested cross-validation and Shapley values. The best-performing predictive models corresponding to six tumor regions and three tumor phenotypes were identified for further comparative analysis. In addition, we designed five experiments with different voxel spacing to assess the sensitivity of the experimental results to the spatial resolution of the voxels.

RESULTS

Our results demonstrated that models based on 2D, 3D, and peritumoral region features yielded mean AUCs and 95% confidence intervals of 0.759 and [0.747-0.771] for lymphovascular invasion, 0.889 and [0.882-0.896] for pleural invasion, and 0.839 and [0.829-0.849] for T-staging in the testing cohort, which was significantly higher than all other models. Similar results were obtained for the model combining the three regional features at five voxel spacings.

CONCLUSION

Our study revealed the predictive role of the developed methods with multi-regional features for the preoperative assessment of prognostic factors in NSCLC. The analysis of different voxel spacing and model interpretability strengthens the experimental findings and contributes to understanding the biological significance of the radiological phenotype.

Cross modality generative learning framework for anatomical transitive Magnetic Resonance Imaging (MRI) from Electrical Impedance Tomography (EIT) image

This paper presents a cross-modality generative learning framework for transitive magnetic resonance imaging (MRI) from electrical impedance tomography (EIT). The proposed framework is aimed at converting low-resolution EIT images to high-resolution wrist MRI images using a cascaded cycle generative adversarial network (CycleGAN) model. This model comprises three main components: the collection of initial EIT from the medical device, the generation of a high-resolution transitive EIT image from the corresponding MRI image for domain adaptation, and the coalescence of two CycleGAN models for cross-modality generation. The initial EIT image was generated at three different frequencies (70 kHz, 140 kHz, and 200 kHz) using a 16-electrode belt. Wrist T1-weighted images were acquired on a 1.5T MRI. A total of 19 normal volunteers were imaged using both EIT and MRI, which resulted in 713 paired EIT and MRI images. The cascaded CycleGAN, end-to-end CycleGAN, and Pix2Pix models were trained and tested on the same cohort. The proposed method achieved the highest accuracy in bone detection, with 0.97 for the proposed cascaded CycleGAN, 0.68 for end-to-end CycleGAN, and 0.70 for the Pix2Pix model. Visual inspection showed that the proposed method reduced bone-related errors in the MRI-style anatomical reference compared with end-to-end CycleGAN and Pix2Pix. Multifrequency EIT inputs reduced the testing normalized root mean squared error of MRI-style anatomical reference from 67.9% ± 12.7% to 61.4% ± 8.8% compared with that of single-frequency EIT. The mean conductivity values of fat and bone from regularized EIT were 0.0435 ± 0.0379 S/m and 0.0183 ± 0.0154 S/m, respectively, when the anatomical prior was employed. These results demonstrate that the proposed framework is able to generate MRI-style anatomical references from EIT images with a good degree of accuracy.

Survey of methods and principles in three-dimensional reconstruction from two-dimensional medical images

Three-dimensional (3D) reconstruction of human organs has gained attention in recent years due to advances in the Internet and graphics processing units. In the coming years, most patient care will shift toward this new paradigm. However, development of fast and accurate 3D models from medical images or a set of medical scans remains a daunting task due to the number of pre-processing steps involved, most of which are dependent on human expertise. In this review, a survey of pre-processing steps was conducted, and reconstruction techniques for several organs in medical diagnosis were studied. Various methods and principles related to 3D reconstruction were highlighted. The usefulness of 3D reconstruction of organs in medical diagnosis was also highlighted.

Deep learning-based affine medical image registration for multimodal minimal-invasive image-guided interventions - A comparative study on generalizability

Multimodal image registration is applied in medical image analysis as it allows the integration of complementary data from multiple imaging modalities. In recent years, various neural network-based approaches for medical image registration have been presented in papers, but due to the use of different datasets, a fair comparison is not possible. In this research 20 different neural networks for an affine registration of medical images were implemented. The networks' performance and the networks' generalizability to new datasets were evaluated using two multimodal datasets - a synthetic and a real patient dataset - of three-dimensional CT and MR images of the liver. The networks were first trained semi-supervised using the synthetic dataset and then evaluated on the synthetic dataset and the unseen patient dataset. Afterwards, the networks were finetuned on the patient dataset and subsequently evaluated on the patient dataset. The networks were compared using our own developed CNN as benchmark and a conventional affine registration with SimpleElastix as baseline. Six networks improved the pre-registration Dice coefficient of the synthetic dataset significantly (p-value < 0.05) and nine networks improved the pre-registration Dice coefficient of the patient dataset significantly and are therefore able to generalize to the new datasets used in our experiments. Many different machine learning-based methods have been proposed for affine multimodal medical image registration, but few are generalizable to new data and applications. It is therefore necessary to conduct further research in order to develop medical image registration techniques that can be applied more widely.

Automatic classification of pulmonary nodules in computed tomography images using pre-trained networks and bag of features

Lung cancer has the highest incidence in the world. The standard tests for its diagnostics are medical imaging exams, sputum cytology, and lung biopsy. Computed Tomography (CT) of the chest plays an essential role in the early detection of nodules since it can allow for more treatment options and increases patient survival. However, the analysis of these exams is a tiring and error-prone process. Thus, computational methods can help the specialist in this analysis. This work addresses the classification of pulmonary nodules as benign or malignant on CT images. Our approach uses the pre-trained VGG16, VGG19, Inception, Resnet50, and Xception, to extract features from each 2D slice of the 3D nodules. Then, we use Principal Component Analysis to reduce the dimensionality of the feature vectors and make them all the same length. Then, we use Bag of Features (BoF) to combine the feature vectors of the different 2D slices and generate only one signature representing the 3D nodule. The classification step uses Random Forest. We evaluated the proposed method with 1,405 segmented nodules from the LIDC-IDRI database and obtained an accuracy of 95.34%, F1-Score of 91.73, kappa of 0.88, sensitivity of 90.53%, specificity of 97.26% and AUC of 0.99. The main conclusion was that the combination by BoF of features extracted from 2D slices using pre-trained architectures produced better results than training 2D and 3D CNNs in the nodules. In addition, the use of BoF also makes the creation of the nodule signature independent of the number of slices.

Querying a Clinical Data Warehouse for Combinations of Clinical and Imaging Data

This study aims to show the feasibility and benefit of single queries in a research data warehouse combining data from a hospital's clinical and imaging systems. We used a comprehensive integration of a production picture archiving and communication system (PACS) with a clinical data warehouse (CDW) for research to create a system that allows data from both domains to be queried jointly with a single query. To achieve this, we mapped the DICOM information model to the extended entity-attribute-value (EAV) data model of a CDW, which allows data linkage and query constraints on multiple levels: the patient, the encounter, a document, and a group level. Accordingly, we have integrated DICOM metadata directly into CDW and linked it to existing clinical data. We included data collected in 2016 and 2017 from the Department of Internal Medicine in this analysis for two query inquiries from researchers targeting research about a disease and in radiology. We obtained quantitative information about the current availability of combinations of clinical and imaging data using a single multilevel query compiled for each query inquiry. We compared these multilevel query results to results that linked data at a single level, resulting in a quantitative representation of results that was up to 112% and 573% higher. An EAV data model can be extended to store data from clinical systems and PACS on multiple levels to enable combined querying with a single query to quickly display actual frequency data.

Multi-agent medical image segmentation: A survey

During the last decades, the healthcare area has increasingly relied on medical imaging for the diagnosis of a growing number of pathologies. The different types of medical images are mostly manually processed by human radiologists for diseases detection and monitoring. However, such a procedure is time-consuming and relies on expert judgment. The latter can be influenced by a variety of factors. One of the most complicated image processing tasks is image segmentation. Medical image segmentation consists of dividing the input image into a set of regions of interest, corresponding to body tissues and organs. Recently, artificial intelligence (AI) techniques brought researchers attention with their promising results for the image segmentation automation. Among AI-based techniques are those that use the Multi-Agent System (MAS) paradigm. This paper presents a comparative study of the multi-agent approaches dedicated to the segmentation of medical images, recently published in the literature.

MediNet: transfer learning approach with MediNet medical visual database

The rapid development of machine learning has increased interest in the use of deep learning methods in medical research. Deep learning in the medical field is used in disease detection and classification problems in the clinical decision-making process. Large amounts of labeled datasets are often required to train deep neural networks; however, in the medical field, the lack of a sufficient number of images in datasets and the difficulties encountered during data collection are among the main problems. In this study, we propose MediNet, a new 10-class visual dataset consisting of Rontgen (X-ray), Computed Tomography (CT), Magnetic Resonance Imaging (MRI), Ultrasound, and Histopathological images such as calcaneal normal, calcaneal tumor, colon benign colon adenocarcinoma, brain normal, brain tumor, breast benign, breast malignant, chest normal, chest pneumonia. AlexNet, VGG19-BN, Inception V3, DenseNet 121, ResNet 101, EfficientNet B0, Nested-LSTM + CNN, and proposed RdiNet deep learning algorithms are used in the transfer learning for pre-training and classification application. Transfer learning aims to apply previously learned knowledge in a new task. Seven algorithms were trained with the MediNet dataset, and the models obtained from these algorithms, namely feature vectors, were recorded. Pre-training models were used for classification studies on chest X-ray images, diabetic retinopathy, and Covid-19 datasets with the transfer learning technique. In performance measurement, an accuracy of 94.84% was obtained in the traditional classification study for the InceptionV3 model in the classification study performed on the Chest X-Ray Images dataset, and the accuracy was increased 98.71% after the transfer learning technique was applied. In the Covid-19 dataset, the classification success of the DenseNet121 model before pre-trained was 88%, while the performance after the transfer application with MediNet was 92%. In the Diabetic retinopathy dataset, the classification success of the Nested-LSTM + CNN model before pre-trained was 79.35%, while the classification success was 81.52% after the transfer application with MediNet. The comparison of results obtained from experimental studies observed that the proposed method produced more successful results.

Graphical abstract

Medical image data augmentation: techniques, comparisons and interpretations

Designing deep learning based methods with medical images has always been an attractive area of research to assist clinicians in rapid examination and accurate diagnosis. Those methods need a large number of datasets including all variations in their training stages. On the other hand, medical images are always scarce due to several reasons, such as not enough patients for some diseases, patients do not want to allow their images to be used, lack of medical equipment or equipment, inability to obtain images that meet the desired criteria. This issue leads to bias in datasets, overfitting, and inaccurate results. Data augmentation is a common solution to overcome this issue and various augmentation techniques have been applied to different types of images in the literature. However, it is not clear which data augmentation technique provides more efficient results for which image type since different diseases are handled, different network architectures are used, and these architectures are trained and tested with different numbers of data sets in the literature. Therefore, in this work, the augmentation techniques used to improve performances of deep learning based diagnosis of the diseases in different organs (brain, lung, breast, and eye) from different imaging modalities (MR, CT, mammography, and fundoscopy) have been examined. Also, the most commonly used augmentation methods have been implemented, and their effectiveness in classifications with a deep network has been discussed based on quantitative performance evaluations. Experiments indicated that augmentation techniques should be chosen carefully according to image types.

CASF-Net: Cross-attention and cross-scale fusion network for medical image segmentation

BACKGROUND

Automatic segmentation of medical images has progressed greatly owing to the development of convolutional neural networks (CNNs). However, there are two uncertainties with current approaches based on convolutional operations: (1) how to eliminate the general limitations that CNNs lack the ability of modeling long-range dependencies and global contextual interactions, and (2) how to efficiently discover and integrate global and local features that are implied in the image. Notably, these two problems are interconnected, yet previous approaches mainly focus on the first problem and ignore the importance of information integration.

METHODS

In this paper, we propose a novel cross-attention and cross-scale fusion network (CASF-Net), which aims to explicitly tap the potential of dual-branch networks and fully integrate the coarse and fine-grained feature representations. Specifically, the well-designed dual-branch encoder hammers at modeling non-local dependencies and multi-scale contexts, significantly improving the quality of semantic segmentation. Moreover, the proposed cross-attention and cross-scale module efficiently perform multi-scale information fusion, being capable of further exploring the long-range contextual information.

RESULTS

Extensive experiments conducted on three different types of medical image segmentation tasks demonstrate the state-of-the-art performance of our proposed method both visually and numerically.

CONCLUSIONS

This paper assembles the feature representation capabilities of CNN and transformer and proposes cross-attention and cross-scale fusion algorithms. The promising results show new possibilities of using cross-fusion mechanisms in more downstream medical image tasks.

Object recognition ability predicts category learning with medical images

We investigated the relationship between category learning and domain-general object recognition ability (o). We assessed this relationship in a radiological context, using a category learning test in which participants judged whether white blood cells were cancerous. In study 1, Bayesian evidence negated a relationship between o and category learning. This lack of correlation occurred despite high reliability in all measurements. However, participants only received feedback on the first 10 of 60 trials. In study 2, we assigned participants to one of two conditions: feedback on only the first 10 trials, or on all 60 trials of the category learning test. We found strong Bayesian evidence for a correlation between o and categorisation accuracy in the full-feedback condition, but not when feedback was limited to early trials. Moderate Bayesian evidence supported a difference between these correlations. Without feedback, participants may stick to simple rules they formulate at the start of category learning, when trials are easier. Feedback may encourage participants to abandon less effective rules and switch to exemplar learning. This work provides the first evidence relating o to a specific learning mechanism, suggesting this ability is more dependent upon exemplar learning mechanisms than rule abstraction. Object-recognition ability could complement other sources of individual differences when predicting accuracy of medical image interpretation.

The effect of choosing optimizer algorithms to improve computer vision tasks: a comparative study

Optimization algorithms are used to improve model accuracy. The optimization process undergoes multiple cycles until convergence. A variety of optimization strategies have been developed to overcome the obstacles involved in the learning process. Some of these strategies have been considered in this study to learn more about their complexities. It is crucial to analyse and summarise optimization techniques methodically from a machine learning standpoint since this can provide direction for future work in both machine learning and optimization. The approaches under consideration include the Stochastic Gradient Descent (SGD), Stochastic Optimization Descent with Momentum, Rung Kutta, Adaptive Learning Rate, Root Mean Square Propagation, Adaptive Moment Estimation, Deep Ensembles, Feedback Alignment, Direct Feedback Alignment, Adfactor, AMSGrad, and Gravity. prove the ability of each optimizer applied to machine learning models. Firstly, tests on a skin cancer using the ISIC standard dataset for skin cancer detection were applied using three common optimizers (Adaptive Moment, SGD, and Root Mean Square Propagation) to explore the effect of the algorithms on the skin images. The optimal training results from the analysis indicate that the performance values are enhanced using the Adam optimizer, which achieved 97.30% accuracy. The second dataset is COVIDx CT images, and the results achieved are 99.07% accuracy based on the Adam optimizer. The result indicated that the utilisation of optimizers such as SGD and Adam improved the accuracy in training, testing, and validation stages.

A Deep Learning and Handcrafted Based Computationally Intelligent Technique for Effective COVID-19 Detection from X-ray/CT-scan Imaging

The world has witnessed dramatic changes because of the advent of COVID19 in the last few days of 2019. During the last more than two years, COVID-19 has badly affected the world in diverse ways. It has not only affected human health and mortality rate but also the economic condition on a global scale. There is an urgent need today to cope with this pandemic and its diverse effects. Medical imaging has revolutionized the treatment of various diseases during the last four decades. Automated detection and classification systems have proven to be of great assistance to the doctors and scientific community for the treatment of various diseases. In this paper, a novel framework for an efficient COVID-19 classification system is proposed which uses the hybrid feature extraction approach. After preprocessing image data, two types of features i.e., deep learning and handcrafted, are extracted. For Deep learning features, two pre-trained models namely ResNet101 and DenseNet201 are used. Handcrafted features are extracted using Weber Local Descriptor (WLD). The Excitation component of WLD is utilized and features are reduced using DCT. Features are extracted from both models, handcrafted features are fused, and significant features are selected using entropy. Experiments have proven the effectiveness of the proposed model. A comprehensive set of experiments have been performed and results are compared with the existing well-known methods. The proposed technique has performed better in terms of accuracy and time.

WatMIF: Multimodal Medical Image Fusion-Based Watermarking for Telehealth Applications

Over recent years, the volume of big data has drastically increased for medical applications. Such data are shared by cloud providers for storage and further processing. Medical images contain sensitive information, and these images are shared with healthcare workers, patients, and, in some scenarios, researchers for diagnostic and study purposes. However, the security of these images in the transfer process is extremely important, especially after the COVID-19 pandemic. This paper proposes a secure watermarking algorithm, termed WatMIF, based on multimodal medical image fusion. The proposed algorithm consists of three major parts: the encryption of the host media, the fusion of multimodal medical images, and the embedding and extraction of the fused mark. We encrypt the host media with a key-based encryption scheme. Then, a nonsubsampled contourlet transform (NSCT)-based fusion scheme is employed to fuse the magnetic resonance imaging (MRI) and computed tomography (CT) scan images to generate the fused mark image. Furthermore, the encrypted host media conceals the fused watermark using redundant discrete wavelet transform (RDWT) and randomised singular value decomposition (RSVD). Finally, denoising convolutional neural network (DnCNN) is used to improve the robustness of the WatMIF algorithm. The simulation experiments on two standard datasets were used to evaluate the algorithm in terms of invisibility, robustness, and security. When compared with the existing algorithms, the robustness is improved by 20.14%. Overall, the implementation of proposed watermarking for hiding fused marks and efficient encryption improved the identity verification, invisibility, robustness and security criteria in our WatMIF algorithm.

Security of medical images for telemedicine: a systematic review

Recently, there has been a rapid growth in the utilization of medical images in telemedicine applications. The authors in this paper presented a detailed discussion of different types of medical images and the attacks that may affect medical image transmission. This survey paper summarizes existing medical data security approaches and the different challenges associated with them. An in-depth overview of security techniques, such as cryptography, steganography, and watermarking are introduced with a full survey of recent research. The objective of the paper is to summarize and assess the different algorithms of each approach based on different parameters such as PSNR, MSE, BER, and NC.

An efficient image encryption scheme for healthcare applications

In recent years, there has been an enormous demand for the security of image multimedia in healthcare organizations. Many schemes have been developed for the security preservation of data in e-health systems however the schemes are not adaptive and cannot resist chosen and known-plaintext attacks. In this contribution, we present an adaptive framework aimed at preserving the security and confidentiality of images transmitted through an e-healthcare system. Our scheme utilizes the 3D-chaotic system to generate a keystream which is used to perform 8-bit and 2-bit permutations of the image. We perform pixel diffusion by a key-image generated using the Piecewise Linear Chaotic Map (PWLCM). We calculate an image parameter using the pixels of the image and perform criss-cross diffusion to enhance security. We evaluate the scheme's performance in terms of histogram analysis, information entropy analysis, statistical analysis, and differential analysis. Using the scheme, we obtain the average Number of Pixels Change Rate (NPCR) and Unified Average Changing Intensity (UACI) values for an image of size 256 × 256 equal to 99.5996 and 33.499 respectively. Furthermore, the average entropy is 7.9971 and the average Peak Signal to Noise Ratio (PSNR) is 7.4756. We further test the scheme on 50 chest X-Ray images of patients having COVID-19 and viral pneumonia and found the average values of variance, PSNR, entropy, and Structural Similarity Index (SSIM) to be 257.6268, 7.7389, 7.9971, and 0.0089 respectively. Furthermore, the scheme generates completely uniform histograms for medical images which reveals that the scheme can resist statistical attacks and can be applied as a security framework in AI-based healthcare.

Unsupervised Deep Anomaly Detection for Medical Images Using an Improved Adversarial Autoencoder

Anomaly detection has been applied in the various disease of medical practice, such as breast cancer, retinal, lung lesion, and skin disease. However, in real-world anomaly detection, there exist a large number of healthy samples, and but very few sick samples. To alleviate the problem of data imbalance in anomaly detection, this paper proposes an unsupervised learning method for deep anomaly detection based on an improved adversarial autoencoder, in which a module called chain of convolutional block (CCB) is employed instead of the conventional skip-connections used in adversarial autoencoder. Such CCB connections provide considerable advantages via direct connections, not only preserving both global and local information but also alleviating the problem of semantic disparity between the encoding features and the corresponding decoding features. The proposed method is thus able to capture the distribution of normal samples within both image space and latent vector space. By means of minimizing the reconstruction error within both spaces during training phase, higher reconstruction error during test phase is indicative of an anomaly. Our method is trained only on the healthy persons in order to learn the distribution of normal samples and can detect sick samples based on high deviation from the distribution of normality in an unsupervised way. Experimental results for multiple datasets from different fields demonstrate that the proposed method yields superior performance to state-of-the-art methods.

Efficient deep learning approach for augmented detection of Coronavirus disease

The new Coronavirus disease 2019 (COVID-19) is rapidly affecting the world population with statistics quickly falling out of date. Due to the limited availability of annotated Coronavirus X-ray and CT images, the detection of COVID-19 remains the biggest challenge in diagnosing this disease. This paper provides a promising solution by proposing a COVID-19 detection system based on deep learning. The proposed deep learning modalities are based on convolutional neural network (CNN) and convolutional long short-term memory (ConvLSTM). Two different datasets are adopted for the simulation of the proposed modalities. The first dataset includes a set of CT images, while the second dataset includes a set of X-ray images. Both of these datasets consist of two categories: COVID-19 and normal. In addition, COVID-19 and pneumonia image categories are classified in order to validate the proposed modalities. The proposed deep learning modalities are tested on both X-ray and CT images as well as a combined dataset that includes both types of images. They achieved an accuracy of 100% and an F1 score of 100% in some cases. The simulation results reveal that the proposed deep learning modalities can be considered and adopted for quick COVID-19 screening.

Towards a better understanding of annotation tools for medical imaging: a survey

Medical imaging refers to several different technologies that are used to view the human body to diagnose, monitor, or treat medical conditions. It requires significant expertise to efficiently and correctly interpret the images generated by each of these technologies, which among others include radiography, ultrasound, and magnetic resonance imaging. Deep learning and machine learning techniques provide different solutions for medical image interpretation including those associated with detection and diagnosis. Despite the huge success of deep learning algorithms in image analysis, training algorithms to reach human-level performance in these tasks depends on the availability of large amounts of high-quality training data, including high-quality annotations to serve as ground-truth. Different annotation tools have been developed to assist with the annotation process. In this survey, we present the currently available annotation tools for medical imaging, including descriptions of graphical user interfaces (GUI) and supporting instruments. The main contribution of this study is to provide an intensive review of the popular annotation tools and show their successful usage in annotating medical imaging dataset to guide researchers in this area.

Automatic method for classifying COVID-19 patients based on chest X-ray images, using deep features and PSO-optimized XGBoost

The COVID-19 pandemic, which originated in December 2019 in the city of Wuhan, China, continues to have a devastating effect on the health and well-being of the global population. Currently, approximately 8.8 million people have already been infected and more than 465,740 people have died worldwide. An important step in combating COVID-19 is the screening of infected patients using chest X-ray (CXR) images. However, this task is extremely time-consuming and prone to variability among specialists owing to its heterogeneity. Therefore, the present study aims to assist specialists in identifying COVID-19 patients from their chest radiographs, using automated computational techniques. The proposed method has four main steps: (1) the acquisition of the dataset, from two public databases; (2) the standardization of images through preprocessing; (3) the extraction of features using a deep features-based approach implemented through the networks VGG19, Inception-v3, and ResNet50; (4) the classifying of images into COVID-19 groups, using eXtreme Gradient Boosting (XGBoost) optimized by particle swarm optimization (PSO). In the best-case scenario, the proposed method achieved an accuracy of 98.71%, a precision of 98.89%, a recall of 99.63%, and an F1-score of 99.25%. In our study, we demonstrated that the problem of classifying CXR images of patients under COVID-19 and non-COVID-19 conditions can be solved efficiently by combining a deep features-based approach with a robust classifier (XGBoost) optimized by an evolutionary algorithm (PSO). The proposed method offers considerable advantages for clinicians seeking to tackle the current COVID-19 pandemic.

An end-to-end approach to segmentation in medical images with CNN and posterior-CRF

Conditional Random Fields (CRFs) are often used to improve the output of an initial segmentation model, such as a convolutional neural network (CNN). Conventional CRF approaches in medical imaging use manually defined features, such as intensity to improve appearance similarity or location to improve spatial coherence. These features work well for some tasks, but can fail for others. For example, in medical image segmentation applications where different anatomical structures can have similar intensity values, an intensity-based CRF may produce incorrect results. As an alternative, we propose Posterior-CRF, an end-to-end segmentation method that uses CNN-learned features in a CRF and optimizes the CRF and CNN parameters concurrently. We validate our method on three medical image segmentation tasks: aorta and pulmonary artery segmentation in non-contrast CT, white matter hyperintensities segmentation in multi-modal MRI, and ischemic stroke lesion segmentation in multi-modal MRI. We compare this with the state-of-the-art CNN-CRF methods. In all applications, our proposed method outperforms the existing methods in terms of Dice coefficient, average volume difference, and lesion-wise F1 score.

Fusing 2D and 3D convolutional neural networks for the segmentation of aorta and coronary arteries from CT images

Automated segmentation of three-dimensional medical images is of great importance for the detection and quantification of certain diseases such as stenosis in the coronary arteries. Many 2D and 3D deep learning models, especially deep convolutional neural networks (CNNs), have achieved state-of-the-art segmentation performance on 3D medical images. Yet, there is a trade-off between the field of view and the utilization of inter-slice information when using pure 2D or 3D CNNs for 3D segmentation, which compromises the segmentation accuracy. In this paper, we propose a two-stage strategy that retains the advantages of both 2D and 3D CNNs and apply the method for the segmentation of the human aorta and coronary arteries, with stenosis, from computed tomography (CT) images. In the first stage, a 2D CNN, which can extract large-field-of-view information, is used to segment the aorta and coronary arteries simultaneously in a slice-by-slice fashion. Then, in the second stage, a 3D CNN is applied to extract the inter-slice information to refine the segmentation of the coronary arteries in certain subregions not resolved well in the first stage. We show that the 3D network of the second stage can improve the continuity between slices and reduce the missed detection rate of the 2D CNN. Compared with directly using a 3D CNN, the two-stage approach can alleviate the class imbalance problem caused by the large non-coronary artery (aorta and background) and the small coronary artery and reduce the training time because the vast majority of negative voxels are excluded in the first stage. To validate the efficacy of our method, extensive experiments are carried out to compare with other approaches based on pure 2D or 3D CNNs and those based on hybrid 2D-3D CNNs.

Artificial intelligence for the early detection of colorectal cancer: A comprehensive review of its advantages and misconceptions

Colorectal cancer (CRC) was the second-ranked worldwide type of cancer during 2020 due to the crude mortality rate of 12.0 per 100000 inhabitants. It can be prevented if glandular tissue (adenomatous polyps) is detected early. Colonoscopy has been strongly recommended as a screening test for both early cancer and adenomatous polyps. However, it has some limitations that include the high polyp miss rate for smaller (< 10 mm) or flat polyps, which are easily missed during visual inspection. Due to the rapid advancement of technology, artificial intelligence (AI) has been a thriving area in different fields, including medicine. Particularly, in gastroenterology AI software has been included in computer-aided systems for diagnosis and to improve the assertiveness of automatic polyp detection and its classification as a preventive method for CRC. This article provides an overview of recent research focusing on AI tools and their applications in the early detection of CRC and adenomatous polyps, as well as an insightful analysis of the main advantages and misconceptions in the field.

Text-based multi-dimensional medical images retrieval according to the features-usage correlation

Emerging medical imaging applications in healthcare, the number and volume of medical images is growing dramatically. Information needs of users in such circumstances, either for clinical or research activities, make the role of powerful medical image search engines more significant. In this paper, a text-based multi-dimensional medical image indexing technique is proposed in which correlation of the features-usages (according to the user's queries) is considered to provide an off-the content indexing while taking users' interestingness into account. Assuming that each medical image has some extracted features (e.g., based on the DICOM standard), correlations of the features are discovered by performing data mining techniques (i.e., quantitative association pattern discovery), on the history of users' queries as a data set. Then, based on the pairwise correlation of the features of medical images (a.k.a. Affinity), set of the all features is fragmented into subsets (using method like the vertical fragmentation of the tables in distribution of relational DBs). After that, each of these subsets of the features turn into a hierarchy of the features (by applying a hierarchical clustering algorithm on that subset), subsequently all of these distinct hierarchies together make a multi-dimensional structure of the features of medical images, which is in fact the proposed text-based (feature-based) multi-dimensional index structure. Constructing and using such text-based multi-dimensional index structure via its specific required operations, medical image retrieval process would be improved in the underlying medical image search engine. Generally, an indexing technique is to provide a logical representation of documents in order to optimize the retrieval process. The proposed indexing technique is designed such that can improve retrieval of medical images in a medical image search engine in terms of its effectiveness and efficiency. Considering correlation of the features of the image would semantically improve precision (effectiveness) of the retrieval process, while traversing them through the hierarchy in one dimension would try to optimize (i.e., minimize) the resources to have a better efficiency. The proposed text-based multi-dimensional indexing technique is implemented using the open source search engine Lucene, and compared with the built-in indexing technique available in the Lucene search engine, and also with the Terrier platform (available for the benchmarking of information retrieval systems) and other the most related indexing techniques. Evaluation results of memory usage and time complexity analysis, beside the experimental evaluations efficiency and effectiveness measures show that the proposed multi-dimensional indexing technique significantly improves both efficiency and effectiveness for a medical image search engine.

Interoperability and Considerations for Standards-Based Exchange of Medical Images: HIMSS-SIIM Collaborative White Paper

This white paper explores the considerations of standards-based interoperability of medical images between organizations, patients, and providers. In this paper, we will look at three different standards-based image exchange implementations that have been deployed to facilitate exchange of images between provider organizations. The paper will describe how each implementation uses applicable technology and standards; the image types that are included; and the governance policies that define participation, access, and trust. Limitations of the solution or non-standard approaches to solve challenges will also be identified. Much can be learned from successes elsewhere, and those learnings will point to recommendations of best practices to facilitate the adoption of image exchange.

Iterative confidence relabeling with deep ConvNets for organ segmentation with partial labels

Training deep ConvNets requires large labeled datasets. However, collecting pixel-level labels for medical image segmentation is very expensive and requires a high level of expertise. In addition, most existing segmentation masks provided by clinical experts focus on specific anatomical structures. In this paper, we propose a method dedicated to handle such partially labeled medical image datasets. We propose a strategy to identify pixels for which labels are correct, and to train Fully Convolutional Neural Networks with a multi-label loss adapted to this context. In addition, we introduce an iterative confidence self-training approach inspired by curriculum learning to relabel missing pixel labels, which relies on selecting the most confident prediction with a specifically designed confidence network that learns an uncertainty measure which is leveraged in our relabeling process. Our approach, INERRANT for Iterative coNfidencE Relabeling of paRtial ANnoTations, is thoroughly evaluated on two public datasets (TCAI and LITS), and one internal dataset with seven abdominal organ classes. We show that INERRANT robustly deals with partial labels, performing similarly to a model trained on all labels even for large missing label proportions. We also highlight the importance of our iterative learning scheme and the proposed confidence measure for optimal performance. Finally we show a practical use case where a limited number of completely labeled data are enriched by publicly available but partially labeled data.

Medical image analysis based on deep learning approach

Medical imaging plays a significant role in different clinical applications such as medical procedures used for early detection, monitoring, diagnosis, and treatment evaluation of various medical conditions. Basicsof the principles and implementations of artificial neural networks and deep learning are essential for understanding medical image analysis in computer vision. Deep Learning Approach (DLA) in medical image analysis emerges as a fast-growing research field. DLA has been widely used in medical imaging to detect the presence or absence of the disease. This paper presents the development of artificial neural networks, comprehensive analysis of DLA, which delivers promising medical imaging applications. Most of the DLA implementations concentrate on the X-ray images, computerized tomography, mammography images, and digital histopathology images. It provides a systematic review of the articles for classification, detection, and segmentation of medical images based on DLA. This review guides the researchers to think of appropriate changes in medical image analysis based on DLA.

A Novel Secure and Robust Encryption Scheme for Medical Images

BACKGROUND

Medical imaging encloses different imaging techniques and processes to image the human body for medical diagnostic and treatment purposes. Hence it plays an important role to improve public health. The technological development in biomedical imaging specifically in X-ray, Computed Tomography (CT), nuclear ultrasound including Positron Emission Tomography (PET), optical and Magnetic Resonance Imaging (MRI) can provide valuable information unique to a person.

OBJECTIVE

In health care applications, the images are needed to be exchanged mostly over a wireless medium. The diagnostic images with confidential information of a patient need to be protected from unauthorized access during transmission. In this paper, a novel encryption method is proposed to improve the security and integrity of medical images.

METHODS

Chaotic map along with DNA cryptography is used for encryption. The proposed method describes a two-phase encryption of medical images.

RESULTS

The performance of the proposed method is also tested by various analysis metrics. The robustness of the method against different noises and attacks is analyzed.

CONCLUSION

The results show that the method is efficient and well suitable for medical images.

Efficient deep learning approach for augmented detection of Coronavirus disease

The new Coronavirus disease 2019 (COVID-19) is rapidly affecting the world population with statistics quickly falling out of date. Due to the limited availability of annotated Coronavirus X-ray and CT images, the detection of COVID-19 remains the biggest challenge in diagnosing this disease. This paper provides a promising solution by proposing a COVID-19 detection system based on deep learning. The proposed deep learning modalities are based on convolutional neural network (CNN) and convolutional long short-term memory (ConvLSTM). Two different datasets are adopted for the simulation of the proposed modalities. The first dataset includes a set of CT images, while the second dataset includes a set of X-ray images. Both of these datasets consist of two categories: COVID-19 and normal. In addition, COVID-19 and pneumonia image categories are classified in order to validate the proposed modalities. The proposed deep learning modalities are tested on both X-ray and CT images as well as a combined dataset that includes both types of images. They achieved an accuracy of 100% and an F1 score of 100% in some cases. The simulation results reveal that the proposed deep learning modalities can be considered and adopted for quick COVID-19 screening.

Computed tomography image representation using the Legendre polynomial and spherical harmonics functions

The representation of computed tomography (CT) images using the Legendre polynomial (LPF) and spherical harmonics (SHF) functions was investigated. We selected 100 two-dimensional (2D) CT images of 10 lung cancer patients and 33 three-dimensional (3D) CT images of head and neck cancer patients. The reproducibility of these special functions was evaluated in terms of the normalized cross-correlation (NCC). For the 2D images, the NCC was 0.990 ± 0.002 (1sd) with an LPF of order 70, whereas for the 3D images, the NCC was 0.971 ± 0.004 (1sd) with an SHF of degree 70. The results showed that the LPF was more efficient than the Fourier series. As the thoracic and head areas are cylindrical and spherical, respectively, expansions with the LPF and SHF achieved an efficient representation of the human body. CT image representation with analytical functions can be potentially beneficial, such as in X-ray scattering estimation.

Medical Image Fusion Method Based on Coupled Neural P Systems in Nonsubsampled Shearlet Transform Domain

Coupled neural P (CNP) systems are a recently developed Turing-universal, distributed and parallel computing model, combining the spiking and coupled mechanisms of neurons. This paper focuses on how to apply CNP systems to handle the fusion of multi-modality medical images and proposes a novel image fusion method. Based on two CNP systems with local topology, an image fusion framework in nonsubsampled shearlet transform (NSST) domain is designed, where the two CNP systems are used to control the fusion of low-frequency NSST coefficients. The proposed fusion method is evaluated on 20 pairs of multi-modality medical images and compared with seven previous fusion methods and two deep-learning-based fusion methods. Quantitative and qualitative experimental results demonstrate the advantage of the proposed fusion method in terms of visual quality and fusion performance.

Kidney segmentation from computed tomography images using deep neural network

BACKGROUND

The precise segmentation of kidneys and kidney tumors can help medical specialists to diagnose diseases and improve treatment planning, which is highly required in clinical practice. Manual segmentation of the kidneys is extremely time-consuming and prone to variability between different specialists due to their heterogeneity. Because of this hard work, computational techniques, such as deep convolutional neural networks, have become popular in kidney segmentation tasks to assist in the early diagnosis of kidney tumors. In this study, we propose an automatic method to delimit the kidneys in computed tomography (CT) images using image processing techniques and deep convolutional neural networks (CNNs) to minimize false positives.

METHODS

The proposed method has four main steps: (1) acquisition of the KiTS19 dataset, (2) scope reduction using AlexNet, (3) initial segmentation using U-Net 2D, and (4) false positive reduction using image processing to maintain the largest elements (kidneys).

RESULTS

The proposed method was evaluated in 210 CTs from the KiTS19 database and obtained the best result with an average Dice coefficient of 96.33%, an average Jaccard index of 93.02%, an average sensitivity of 97.42%, an average specificity of 99.94% and an average accuracy of 99.92%. In the KiTS19 challenge, it presented an average Dice coefficient of 93.03%.

CONCLUSION

In our method, we demonstrated that the kidney segmentation problem in CT can be solved efficiently using deep neural networks to define the scope of the problem and segment the kidneys with high precision and with the use of image processing techniques to reduce false positives.

Unlocking the PACS DICOM Domain for its Use in Clinical Research Data Warehouses

Clinical Data Warehouses (DWHs) are used to provide researchers with simplified access to pseudonymized and homogenized clinical routine data from multiple primary systems. Experience with the integration of imaging and metadata from picture archiving and communication systems (PACS), however, is rare. Our goal was therefore to analyze the viability of integrating a production PACS with a research DWH to enable DWH queries combining clinical and medical imaging metadata and to enable the DWH to display and download images ad hoc. We developed an application interface that enables to query the production PACS of a large hospital from a clinical research DWH containing pseudonymized data. We evaluated the performance of bulk extracting metadata from the PACS to the DWH and the performance of retrieving images ad hoc from the PACS for display and download within the DWH. We integrated the system into the query interface of our DWH and used it successfully in four use cases. The bulk extraction of imaging metadata required a median (quartiles) time of 0.09 (0.03–2.25) to 12.52 (4.11–37.30) seconds for a median (quartiles) number of 10 (3–29) to 103 (8–693) images per patient, depending on the extraction approach. The ad hoc image retrieval from the PACS required a median (quartiles) of 2.57 (2.57–2.79) seconds per image for the download, but 5.55 (4.91–6.06) seconds to display the first and 40.77 (38.60–41.63) seconds to display all images using the pure web-based viewer. A full integration of a production PACS with a research DWH is viable and enables various use cases in research. While the extraction of basic metadata from all images can be done with reasonable effort, the extraction of all metadata seems to be more appropriate for subgroups.

Breast cancer diagnosis from histopathological images using textural features and CBIR

Currently, breast cancer diagnosis is an extensively researched topic. An effective method to diagnose breast cancer is to use histopathological images. However, extracting features from these images is a challenging task. Thus, we propose a method that uses phylogenetic diversity indexes to characterize images for creating a model to classify histopathological breast images into four classes - invasive carcinoma, in situ carcinoma, normal tissue, and benign lesion. The classifiers used were the most robust ones according to the existing literature: XGBoost, random forest, multilayer perceptron, and support vector machine. Moreover, we performed content-based image retrieval to confirm the classification results and suggest a ranking for sets of images that were not labeled. The results obtained were considerably robust and proved to be effective for the composition of a CADx system to help specialists at large medical centers.

[Role of artificial intelligence in the diagnosis and treatment of gastrointestinal diseases]

The rapid development of computer technologies brings us great changes in daily life and work. Artificial intelligence is a branch of computer science, which is to allow computers to exercise activities that are normally confined to intelligent life. The broad sense of artificial intelligence includes machine learning and robots. This article mainly focuses on machine learning and related medical fields, and deep learning is an artificial neural network in machine learning. Convolutional neural network (CNN) is a type of deep neural network, that is developed on the basis of deep neural network, further imitating the structure of the visual cortex of the brain and the principle of visual activity. The current machine learning method used in medical big data analysis is mainly CNN. In the next few years, it is the developing trend that artificial intelligence as a conventional tool will enter the relevant departments of medical image interpretation. In addition, this article also shares the progress of the integration of artificial intelligence and biomedicine combined with actual cases, and mainly introduces the current status of CNN application research in pathological diagnosis, imaging diagnosis and endoscopic diagnosis for gastrointestinal diseases.

Dataset of breast ultrasound images

Breast cancer is one of the most common causes of death among women worldwide. Early detection helps in reducing the number of early deaths. The data presented in this article reviews the medical images of breast cancer using ultrasound scan. Breast Ultrasound Dataset is categorized into three classes: normal, benign, and malignant images. Breast ultrasound images can produce great results in classification, detection, and segmentation of breast cancer when combined with machine learning.

The quantization error in a Self-Organizing Map as a contrast and colour specific indicator of single-pixel change in large random patterns

The quantization error in a fixed-size Self-Organizing Map (SOM) with unsupervised winner-take-all learning has previously been used successfully to detect, in minimal computation time, highly meaningful changes across images in medical time series and in time series of satellite images. Here, the functional properties of the quantization error in SOM are explored further to show that the metric is capable of reliably discriminating between the finest differences in local contrast intensities and contrast signs. While this capability of the QE is akin to functional characteristics of a specific class of retinal ganglion cells (the so-called Y-cells) in the visual systems of the primate and the cat, the sensitivity of the QE surpasses the capacity limits of human visual detection. Here, the quantization error in the SOM is found to reliably signal changes in contrast or colour when contrast information is removed from or added to the image, but not when the amount and relative weight of contrast information is constant and only the local spatial position of contrast elements in the pattern changes. While the RGB Mean reflects coarser changes in colour or contrast well enough, the SOM-QE is shown to outperform the RGB Mean in the detection of single-pixel changes in images with up to five million pixels. This could have important implications in the context of unsupervised image learning and computational building block approaches to large sets of image data (big data), including deep learning blocks, and automatic detection of contrast change at the nanoscale in Transmission or Scanning Electron Micrographs (TEM, SEM), or at the subpixel level in multispectral and hyper-spectral imaging data.

The quantization error in a Self-Organizing Map as a contrast and colour specific indicator of single-pixel change in large random patterns

The quantization error in a fixed-size Self-Organizing Map (SOM) with unsupervised winner-take-all learning has previously been used successfully to detect, in minimal computation time, highly meaningful changes across images in medical time series and in time series of satellite images. Here, the functional properties of the quantization error in SOM are explored further to show that the metric is capable of reliably discriminating between the finest differences in local contrast intensities and contrast signs. While this capability of the QE is akin to functional characteristics of a specific class of retinal ganglion cells (the so-called Y-cells) in the visual systems of the primate and the cat, the sensitivity of the QE surpasses the capacity limits of human visual detection. Here, the quantization error in the SOM is found to reliably signal changes in contrast or colour when contrast information is removed from or added to the image, but not when the amount and relative weight of contrast information is constant and only the local spatial position of contrast elements in the pattern changes. While the RGB Mean reflects coarser changes in colour or contrast well enough, the SOM-QE is shown to outperform the RGB Mean in the detection of single-pixel changes in images with up to five million pixels. This could have important implications in the context of unsupervised image learning and computational building block approaches to large sets of image data (big data), including deep learning blocks, and automatic detection of contrast change at the nanoscale in Transmission or Scanning Electron Micrographs (TEM, SEM), or at the subpixel level in multispectral and hyper-spectral imaging data.