Endowing a Content-Based Medical Image Retrieval System with Perceptual Similarity Using Ensemble Strategy

Dynamic distance learning for joint assessment of visual and semantic similarities within the framework of medical image retrieval

The similarity measure is an essential part of medical image retrieval systems for assisting in radiological diagnosis. Attempts have been made to use distance metric learning approaches to improve the retrieval performance while decreasing the semantic gap. However, existing approaches did not resolve the problem of dependency between images (e.g. normal and abnormal images are compared with the same distance). This affects the semantic and the visual similarity. Thus, this work aims at learning a distance metric which preserves both visual resemblance and semantic similarity and modeling this distance in order to treat each query independently. The proposed method is described in three stages: (1) low-level image feature extraction, (2) offline distance metric modeling, and (3) online retrieval. The first stage exploits transform-domain texture descriptors based on local binary pattern histogram Fourier, shearlet, and curvelet transforms. The second stage is carried out using low-level features and machine learning. Given a query image, the online retrieval is based on the evaluation of the similarity between this image and each image within the dataset, while using a distance that is dynamically defined according to the query image. Realized experiments on the challenging Mammographic Image Analysis Society (MIAS) and Digital Database for Screening Mammography (DDSM) datasets prove the effectiveness of the proposed method in determining dynamically the adequate distance and retrieving the most semantically similar images, while investigating single low-level features as well as fused ones.

Effective Diagnosis and Treatment through Content-Based Medical Image Retrieval (CBMIR) by Using Artificial Intelligence

Medical-image-based diagnosis is a tedious task‚ and small lesions in various medical images can be overlooked by medical experts due to the limited attention span of the human visual system, which can adversely affect medical treatment. However, this problem can be resolved by exploring similar cases in the previous medical database through an efficient content-based medical image retrieval (CBMIR) system. In the past few years, heterogeneous medical imaging databases have been growing rapidly with the advent of different types of medical imaging modalities. Recently, a medical doctor usually refers to various types of imaging modalities all together such as computed tomography (CT), magnetic resonance imaging (MRI), X-ray, and ultrasound, etc of various organs in order for the diagnosis and treatment of specific disease. Accurate classification and retrieval of multimodal medical imaging data is the key challenge for the CBMIR system. Most previous attempts use handcrafted features for medical image classification and retrieval, which show low performance for a massive collection of multimodal databases. Although there are a few previous studies on the use of deep features for classification, the number of classes is very small. To solve this problem, we propose the classification-based retrieval system of the multimodal medical images from various types of imaging modalities by using the technique of artificial intelligence, named as an enhanced residual network (ResNet). Experimental results with 12 databases including 50 classes demonstrate that the accuracy and F1.score by our method are respectively 81.51% and 82.42% which are higher than those by the previous method of CBMIR (the accuracy of 69.71% and F1.score of 69.63%).

An overview of deep learning in medical imaging focusing on MRI

What has happened in machine learning lately, and what does it mean for the future of medical image analysis? Machine learning has witnessed a tremendous amount of attention over the last few years. The current boom started around 2009 when so-called deep artificial neural networks began outperforming other established models on a number of important benchmarks. Deep neural networks are now the state-of-the-art machine learning models across a variety of areas, from image analysis to natural language processing, and widely deployed in academia and industry. These developments have a huge potential for medical imaging technology, medical data analysis, medical diagnostics and healthcare in general, slowly being realized. We provide a short overview of recent advances and some associated challenges in machine learning applied to medical image processing and image analysis. As this has become a very broad and fast expanding field we will not survey the entire landscape of applications, but put particular focus on deep learning in MRI. Our aim is threefold: (i) give a brief introduction to deep learning with pointers to core references; (ii) indicate how deep learning has been applied to the entire MRI processing chain, from acquisition to image retrieval, from segmentation to disease prediction; (iii) provide a starting point for people interested in experimenting and perhaps contributing to the field of deep learning for medical imaging by pointing out good educational resources, state-of-the-art open-source code, and interesting sources of data and problems related medical imaging.

Overview on subjective similarity of images for content-based medical image retrieval

Computer-aided diagnosis systems for assisting the classification of various diseases have the potential to improve radiologists' diagnostic accuracy and efficiency, as reported in several studies. Conventional systems generally provide the probabilities of disease types in terms of numerical values, a method that may not be efficient for radiologists who are trained by reading a large number of images. Presentation of reference images similar to those of a new case being diagnosed can supplement the probability outputs based on computerized analysis as an intuitive guide, and it can assist radiologists in their diagnosis, reporting, and treatment planning. Many studies on content-based medical image retrievals have been reported on. For retrieval of perceptually similar and diagnostically relevant images, incorporation of perceptual similarity data by radiologists has been suggested. In this paper, studies on image retrieval methods are reviewed with a special focus on quantification, utilization, and the evaluation of subjective similarities between pairs of images.

A similarity measure method combining location feature for mammogram retrieval

BACKGROUND

Breast cancer, the most common malignancy among women, has a high mortality rate in clinical practice. Early detection, diagnosis and treatment can reduce the mortalities of breast cancer greatly. The method of mammogram retrieval can help doctors to find the early breast lesions effectively and determine a reasonable feature set for image similarity measure. This will improve the accuracy effectively for mammogram retrieval.

METHODS

This paper proposes a similarity measure method combining location feature for mammogram retrieval. Firstly, the images are pre-processed, the regions of interest are detected and the lesions are segmented in order to get the center point and radius of the lesions. Then, the method, namely Coherent Point Drift, is used for image registration with the pre-defined standard image. The center point and radius of the lesions after registration are obtained and the standard location feature of the image is constructed. This standard location feature can help figure out the location similarity between the image pair from the query image to each dataset image in the database. Next, the content feature of the image is extracted, including the Histogram of Oriented Gradients, the Edge Direction Histogram, the Local Binary Pattern and the Gray Level Histogram, and the image pair content similarity can be calculated using the Earth Mover's Distance. Finally, the location similarity and content similarity are fused to form the image fusion similarity, and the specified number of the most similar images can be returned according to it.

RESULTS

In the experiment, 440 mammograms, which are from Chinese women in Northeast China, are used as the database. When fusing 40% lesion location feature similarity and 60% content feature similarity, the results have obvious advantages. At this time, precision is 0.83, recall is 0.76, comprehensive indicator is 0.79, satisfaction is 96.0%, mean is 4.2 and variance is 17.7.

CONCLUSIONS

The results show that the precision and recall of this method have obvious advantage, compared with the content-based image retrieval.

Automatic weighing attribute to retrieve similar lung cancer nodules

BACKGROUND

Cancer is a disease characterized as an uncontrolled growth of abnormal cells that invades neighboring tissues and destroys them. Lung cancer is the primary cause of cancer-related deaths in the world, and it diagnosis is a complex task for specialists and it presents some big challenges as medical image interpretation process, pulmonary nodule detection and classification. In order to aid specialists in the early diagnosis of lung cancer, computer assistance must be integrated in the imaging interpretation and pulmonary nodule classification processes. Methods of Content-Based Image Retrieval (CBIR) have been described as one promising technique to computer-aided diagnosis and is expected to aid radiologists on image interpretation with a second opinion. However, CBIR presents some limitations: image feature extraction process and appropriate similarity measure. The efficiency of CBIR systems depends on calculating image features that may be relevant to the case similarity analysis. When specialists classify a nodule, they are supported by information from exams, images, etc. But each information has more or less weight over decision making about nodule malignancy. Thus, finding a way to measure the weight allows improvement of image retrieval process through the assignment of higher weights to that attributes that best characterize the nodules.

METHODS

In this context, the aim of this work is to present a method to automatically calculate attribute weights based on local learning to reflect the interpretation on image retrieval process. The process consists of two stages that are performed sequentially and cyclically: Evaluation Stage and Training Stage. At each iteration the weights are adjusted according to retrieved nodules. After some iterations, it is possible reach a set of attribute weights that optimize the recovery of similar nodes.

RESULTS

The results achieved by updated weights were promising because was possible increase precision by 10% to 6% on average to retrieve of benign and malignant nodules, respectively, with recall of 25% compared with tests without weights associated to attributes in similarity metric. The best result, we reaching values over 100% of precision average until thirtieth lung cancer nodule retrieved.

CONCLUSIONS

Based on the results, WED applied to the three vectors used attributes (3D TA, 3D MSA and InV), with weights adjusted by the process, always achieved better results than those found with ED. With the weights, the Precision was increased on average by 17.3% compared with using ED.