Computer-aided diagnosis of mammographic masses using scalable image retrieval

Multiple semantic X-ray medical image retrieval using efficient feature vector extracted by FPN

OBJECTIVE

Content-based medical image retrieval (CBMIR) has become an important part of computer-aided diagnostics (CAD) systems. The complex medical semantic information inherent in medical images is the most difficult part to improve the accuracy of image retrieval. Highly expressive feature vectors play a crucial role in the search process. In this paper, we propose an effective deep convolutional neural network (CNN) model to extract concise feature vectors for multiple semantic X-ray medical image retrieval.

METHODS

We build a feature pyramid based CNN model with ResNet50V2 backbone to extract multi-level semantic information. And we use the well-known public multiple semantic annotated X-ray medical image data set IRMA to train and test the proposed model.

RESULTS

Our method achieves an IRMA error of 32.2, which is the best score compared to the existing literature on this dataset.

CONCLUSIONS

The proposed CNN model can effectively extract multi-level semantic information from X-ray medical images. The concise feature vectors can improve the retrieval accuracy of multi-semantic and unevenly distributed X-ray medical images.

A novel deep neural network with adaptive sine cosine crow search (DNN-ASCCS) model for content based medical image reterival

Tumor is the second major cause of death in women worldwide. Breast cancer diagnosis and treatment can be difficult for radiologists. As a result, primary care helps to avoid disease and mortality. The study’s main goal is to improve treatment choices and to save lives by detecting breast cancer earlier. For classification problems, we propose a DNN-ASCC architecture in this study. The Fast Non-Local Means Filter completes the initial preprocessing stage. The binary grasshopper optimization algorithm (BGOA) and the grey-level run length matrix are utilized to choose the best features for the feature extraction operation. The suggested hybrid classifier (DNN-ASCCS) is critical for identifying normal and malignant tumors. Breast cancer is accurately detected by the suggested hybrid classifier. The recommended (DNN-ASCCS) was developed using MATLAB and datasets from the BIDCIDRI. The results of the simulation showed that the proposed technique has an accurate results in classification (99.17 percent) and robustness analysis is also done. When compared to alternative approaches, experimental results show that the suggested method is efficient.

Precision medical image hash retrieval by interpretability and feature fusion

BACKGROUND AND OBJECTIVE

To address the problem of low accuracy of medical image retrieval due to high inter-class similarity and easy omission of lesions, a precision medical image hash retrieval method combining interpretability and feature fusion is proposed, taking chest X-ray images as an example.

METHODS

Firstly, the DenseNet-121 network is pre-trained on a large dataset of medical images without manual annotation using the comparison to learn (C2L) method to obtain a backbone network model containing more medical representations with training weights. Then, a global network is constructed by using global image learning to acquire an interpretable saliency map as attention mechanisms, which can generate a mask crop to get a local discriminant region. Thirdly, the local discriminant regions are used as local network inputs to obtain local features, and the global features are used with the local features by dimension in the pooling layer. Finally, a hash layer is added between the fully connected layer and the classification layer of the backbone network, defining classification loss, quantization loss and bit-balanced loss functions to generate high-quality hash codes. The final retrieval result is output by calculating the similarity metric of the hash codes.

RESULTS

Experiments on the Chest X-ray8 dataset demonstrate that our proposed interpretable saliency map can effectively locate focal regions, the fusion of features can avoid information omission, and the combination of three loss functions can generate more accurate hash codes. Compared with the current advanced medical image retrieval methods, this method can effectively improve the accuracy of medical image retrieval.

CONCLUSIONS

The proposed hash retrieval approach combining interpretability and feature fusion can effectively improve the accuracy of medical image retrieval which can be potentially applied in computer-aided-diagnosis systems.

C-NMC: B-lineage acute lymphoblastic leukaemia: A blood cancer dataset

Development of computer-aided cancer diagnostic tools is an active research area owing to the advancements in deep-learning domain. Such technological solutions provide affordable and easily deployable diagnostic tools. Leukaemia, or blood cancer, is one of the leading cancers causing more than 0.3 million deaths every year. In order to aid the development of such an AI-enabled tool, we collected and curated a microscopic image dataset, namely C-NMC, of more than 15000 cancer cell images at a very high resolution of B-Lineage Acute Lymphoblastic Leukaemia (B-ALL). The dataset is prepared at the subject-level and contains images of both healthy and cancer patients. So far, this is the largest (as well as curated) dataset on B-ALL cancer in the public domain. C-NMC is available at The Cancer Imaging Archive (TCIA), USA and can be helpful for the research community worldwide for the development of B-ALL cancer diagnostic tools. This dataset was utilized in an international medical imaging challenge held at ISBI 2019 conference in Venice, Italy. In this paper, we present a detailed description and challenges of this dataset. We also present benchmarking results of all the methods applied so far on this dataset.

A Bottom-Up Review of Image Analysis Methods for Suspicious Region Detection in Mammograms

Breast cancer is one of the most common death causes amongst women all over the world. Early detection of breast cancer plays a critical role in increasing the survival rate. Various imaging modalities, such as mammography, breast MRI, ultrasound and thermography, are used to detect breast cancer. Though there is a considerable success with mammography in biomedical imaging, detecting suspicious areas remains a challenge because, due to the manual examination and variations in shape, size, other mass morphological features, mammography accuracy changes with the density of the breast. Furthermore, going through the analysis of many mammograms per day can be a tedious task for radiologists and practitioners. One of the main objectives of biomedical imaging is to provide radiologists and practitioners with tools to help them identify all suspicious regions in a given image. Computer-aided mass detection in mammograms can serve as a second opinion tool to help radiologists avoid running into oversight errors. The scientific community has made much progress in this topic, and several approaches have been proposed along the way. Following a bottom-up narrative, this paper surveys different scientific methodologies and techniques to detect suspicious regions in mammograms spanning from methods based on low-level image features to the most recent novelties in AI-based approaches. Both theoretical and practical grounds are provided across the paper sections to highlight the pros and cons of different methodologies. The paper's main scope is to let readers embark on a journey through a fully comprehensive description of techniques, strategies and datasets on the topic.

Thermography as an Economical Alternative Modality to Mammography for Early Detection of Breast Cancer

Breast cancer has become a menacing form of cancer among women accounting for 11.6% of total deaths of 9.6 million due to all types of cancer every year all over the world. Early detection increases chances of survival and reduces the cost of treatment as well. Screening modalities such as mammography or thermography are used to detect cancer early; thus, several lives can be saved with timely treatment. But, there are interpretational failures on the part of the radiologists to read the mammograms or thermograms and also there are interobservational and intraobservational differences between them. So, the degree of variations among the different radiologists in the interpretation of results is very high resulting in false positives and false negatives. The double reading can reduce the human errors involved in the interpretation of mammograms. But, the limited number of medical professionals in developing or underdeveloped countries puts a limitation on this remedial way. So, a computer-aided system (CAD) is proposed to detect the benign cases from the abnormal cases that can result in automatic detection of breast cancer or can provide a double reading in the case of nonavailability of the trained medical professionals in developing economies. The generally accepted screening modality is mammography for the early detection of cancer. But thermography has been tried for early detection of breast cancer in recent times. The high metabolic activity of the cancer cells results in an early change in the temperature profile of the region. This shows asymmetry between normal and cancerous breast which can be detected using different techniques. Thus, this work is focussed on the use of thermography in the early detection of breast cancer. An experimental study is conducted to find the results of classification accuracy to compare the efficacy of thermography and mammography in classifying the normal from abnormal ones and further abnormal ones into benign and malignant cases. Thermography is found to have classification accuracy almost at par with mammography for classifying the cancerous breasts from healthy ones with classification accuracies of thermography and mammography being 96.57% and 98.11%, respectively. Thermography is found to have much better accuracy in identifying benign cases from the malignant ones with the classification accuracy of 92.70% as compared to 82.05% with mammography. This will result in the early detection of cancer. The advantage of being portable and inexpensive makes thermography an attractive modality to be used in economically backward rural areas where mammography is not practically possible.

Medical Image Retrieval Using Empirical Mode Decomposition with Deep Convolutional Neural Network

Content-based medical image retrieval (CBMIR) systems attempt to search medical image database to narrow the semantic gap in medical image analysis. The efficacy of high-level medical information representation using features is a major challenge in CBMIR systems. Features play a vital role in the accuracy and speed of the search process. In this paper, we propose a deep convolutional neural network- (CNN-) based framework to learn concise feature vector for medical image retrieval. The medical images are decomposed into five components using empirical mode decomposition (EMD). The deep CNN is trained in a supervised way with multicomponent input, and the learned features are used to retrieve medical images. The IRMA dataset, containing 11,000 X-ray images, 116 classes, is used to validate the proposed method. We achieve a total IRMA error of 43.21 and a mean average precision of 0.86 for retrieval task and IRMA error of 68.48 and F1 measure of 0.66 on classification task, which is the best result compared with existing literature for this dataset.

Whole mammographic mass segmentation using attention mechanism and multiscale pooling adversarial network

Purpose: Since breast mass is a clear sign of breast cancer, its precise segmentation is of great significance for the diagnosis of breast cancer. However, the current diagnosis relies mainly on radiologists who spend time extracting features manually, which inevitably reduces the efficiency of diagnosis. Therefore, designing an automatic segmentation method is urgently necessary for the accurate segmentation of breast masses. Approach: We propose an effective attention mechanism and multiscale pooling conditional generative adversarial network (AM-MSP-cGAN), which accurately achieves mass automatic segmentation in whole mammograms. In AM-MSP-cGAN, U-Net is utilized as a generator network by incorporating attention mechanism (AM) into it, which allows U-Net to pay more attention to the target mass regions without additional cost. As a discriminator network, a convolutional neural network with multiscale pooling module is used to learn more meticulous features from the masses with rough and fuzzy boundaries. The proposed model is trained and tested on two public datasets: CBIS-DDSM and INbreast. Results: Compared with other state-of-the-art methods, the AM-MSP-cGAN can achieve better segmentation results in terms of the dice similarity coefficient (Dice) and Hausdorff distance metrics, achieving top scores of 84.49% and 5.01 on CBIS-DDSM, and 83.92% and 5.81 on INbreast, respectively. Therefore, qualitative and quantitative experiments illustrate that the proposed model is effective and robust for the mass segmentation in whole mammograms. Conclusions: The proposed deep learning model is suitable for the automatic segmentation of breast masses, which provides technical assistance for subsequent pathological structure analysis.

A disease category feature database construction method of brain image based on deep convolutional neural network

BACKGROUND

Constructing a medical image feature database according to the category of disease can achieve a quick retrieval of images with similar pathological features. Therefore, this approach has important application values in the fields such as auxiliary diagnosis, teaching, research, and telemedicine.

METHODS

Based on the deep convolutional neural network, an image classifier applicable to brain disease was designed to distinguish between the image features of the different brain diseases with similar anatomical structures. Through the extraction and analysis of visual features, the images were labelled with the corresponding semantic features of a specific disease category, which can establish an association between the visual features of brain images and the semantic features of the category of disease which will permit to construct a disease category feature database of brain images.

RESULTS

Based on the similarity measurement and the matching strategy of high-dimensional visual feature, a high-precision retrieval of brain image with semantics category was achieved, and the constructed disease category feature database of brain image was tested and evaluated through large numbers of pathological image retrieval experiments, the accuracy and the effectiveness of the proposed approach was verified.

CONCLUSION

The disease category feature database of brain image constructed by the proposed approach achieved a quick and effective retrieval of images with similar pathological features, which is beneficial to the categorization and analysis of intractable brain diseases. This provides an effective application tool such as case-based image data management, evidence-based medicine and clinical decision support.

Multiple Query Content-Based Image Retrieval Using Relevance Feature Weight Learning

We propose a novel multiple query retrieval approach, named weight-learner, which relies on visual feature discrimination to estimate the distances between the query images and images in the database. For each query image, this discrimination consists of learning, in an unsupervised manner, the optimal relevance weight for each visual feature/descriptor. These feature relevance weights are designed to reduce the semantic gap between the extracted visual features and the user’s high-level semantics. We mathematically formulate the proposed solution through the minimization of some objective functions. This optimization aims to produce optimal feature relevance weights with respect to the user query. The proposed approach is assessed using an image collection from the Corel database.

A multi-feature image retrieval scheme for pulmonary nodule diagnosis

Deep analysis of radiographic images can quantify the extent of intra-tumoral heterogeneity for personalized medicine.In this paper, we propose a novel content-based multi-feature image retrieval (CBMFIR) scheme to discriminate pulmonary nodules benign or malignant. Two types of features are applied to represent the pulmonary nodules. With each type of features, a single-feature distance metric model is proposed to measure the similarity of pulmonary nodules. And then, multiple single-feature distance metric models learned from different types of features are combined to a multi-feature distance metric model. Finally, the learned multi-feature distance metric is used to construct a content-based image retrieval (CBIR) scheme to assist the doctors in diagnosis of pulmonary nodules. The classification accuracy and retrieval accuracy are used to evaluate the performance of the scheme.The classification accuracy is 0.955 ± 0.010, and the retrieval accuracies outperform the comparison methods.The proposed CBMFIR scheme is effective in diagnosis of pulmonary nodules. Our method can better integrate multiple types of features from pulmonary nodules.

Classification of radiographic lung pattern based on texture analysis and machine learning

This study evaluated the feasibility of using texture analysis and machine learning to distinguish radiographic lung patterns. A total of 1200 regions of interest (ROIs) including four specific lung patterns (normal, alveolar, bronchial, and unstructured interstitial) were obtained from 512 thoracic radiographs of 252 dogs and 65 cats. Forty-four texture parameters based on eight methods of texture analysis (first-order statistics, spatial gray-level-dependence matrices, gray-level-difference statistics, gray-level run length image statistics, neighborhood gray-tone difference matrices, fractal dimension texture analysis, Fourier power spectrum, and Law's texture energy measures) were used to extract textural features from the ROIs. The texture parameters of each lung pattern were compared and used for training and testing of artificial neural networks. Classification performance was evaluated by calculating accuracy and the area under the receiver operating characteristic curve (AUC). Forty texture parameters showed significant differences between the lung patterns. The accuracy of lung pattern classification was 99.1% in the training dataset and 91.9% in the testing dataset. The AUCs were above 0.98 in the training set and above 0.92 in the testing dataset. Texture analysis and machine learning algorithms may potentially facilitate the evaluation of medical images.

Breast Cancer Detection and Diagnosis Using Mammographic Data: Systematic Review

BACKGROUND

Machine learning (ML) has become a vital part of medical imaging research. ML methods have evolved over the years from manual seeded inputs to automatic initializations. The advancements in the field of ML have led to more intelligent and self-reliant computer-aided diagnosis (CAD) systems, as the learning ability of ML methods has been constantly improving. More and more automated methods are emerging with deep feature learning and representations. Recent advancements of ML with deeper and extensive representation approaches, commonly known as deep learning (DL) approaches, have made a very significant impact on improving the diagnostics capabilities of the CAD systems.

OBJECTIVE

This review aimed to survey both traditional ML and DL literature with particular application for breast cancer diagnosis. The review also provided a brief insight into some well-known DL networks.

METHODS

In this paper, we present an overview of ML and DL techniques with particular application for breast cancer. Specifically, we search the PubMed, Google Scholar, MEDLINE, ScienceDirect, Springer, and Web of Science databases and retrieve the studies in DL for the past 5 years that have used multiview mammogram datasets.

RESULTS

The analysis of traditional ML reveals the limited usage of the methods, whereas the DL methods have great potential for implementation in clinical analysis and improve the diagnostic capability of existing CAD systems.

CONCLUSIONS

From the literature, it can be found that heterogeneous breast densities make masses more challenging to detect and classify compared with calcifications. The traditional ML methods present confined approaches limited to either particular density type or datasets. Although the DL methods show promising improvements in breast cancer diagnosis, there are still issues of data scarcity and computational cost, which have been overcome to a significant extent by applying data augmentation and improved computational power of DL algorithms.

Intelligent Measurement of Spinal Curvature Using Cascade Gentle AdaBoost Classifier and Region-Based DRLSE

For spinal curvature measurements, because of the anatomical complexity of the spine CT image, developing an automated method to avoid manual landmark is a challenging task. In this study, we propose an intelligent framework that integrates the cascade AdaBoost classifier and region-based distance regularized level set evolution (DRLSE) with the vertebral centroid measurement. First, the histogram-of-oriented-gradients based cascade gentle AdaBoost classifier is used to detect automatically and localize vertebral bodies from computer tomography (CT) spinal images. Considering these vertebral pathological images enables us to produce a diverse training dataset. Then, the DRLSE method introduces the local region information to converge the vertebral boundary quickly. The located bounding box is regarded as an accurate initial contour. This avoids the negative impact of manual initialization. Finally, we perform vertebral centroid extraction and spinal curve fitting. The spinal curvature angle is determined by calculating the angle between two tangents to the curve. We verified the effectiveness of the proposed method on 10 spine CT volumes. Quantitative comparison against the ground-truth centroids yielded a detection accuracy rate of 98.3% and a mean centroid location error of 1.15 mm. The comparative results with existing methods demonstrate that the proposed method can accurately detect and segment vertebral bodies. Furthermore, the spinal curvature can be automatically measured without manual landmark.

A Technical Review of Convolutional Neural Network-Based Mammographic Breast Cancer Diagnosis

This study reviews the technique of convolutional neural network (CNN) applied in a specific field of mammographic breast cancer diagnosis (MBCD). It aims to provide several clues on how to use CNN for related tasks. MBCD is a long-standing problem, and massive computer-aided diagnosis models have been proposed. The models of CNN-based MBCD can be broadly categorized into three groups. One is to design shallow or to modify existing models to decrease the time cost as well as the number of instances for training; another is to make the best use of a pretrained CNN by transfer learning and fine-tuning; the third is to take advantage of CNN models for feature extraction, and the differentiation of malignant lesions from benign ones is fulfilled by using machine learning classifiers. This study enrolls peer-reviewed journal publications and presents technical details and pros and cons of each model. Furthermore, the findings, challenges and limitations are summarized and some clues on the future work are also given. Conclusively, CNN-based MBCD is at its early stage, and there is still a long way ahead in achieving the ultimate goal of using deep learning tools to facilitate clinical practice. This review benefits scientific researchers, industrial engineers, and those who are devoted to intelligent cancer diagnosis.

Deep Convolutional Hashing for Low-Dimensional Binary Embedding of Histopathological Images

Compact binary representations of histopa-thology images using hashing methods provide efficient approximate nearest neighbor search for direct visual query in large-scale databases. They can be utilized to measure the probability of the abnormality of the query image based on the retrieved similar cases, thereby providing support for medical diagnosis. They also allow for efficient managing of large-scale image databases because of a low storage requirement. However, the effectiveness of binary representations heavily relies on the visual descriptors that represent the semantic information in the histopathological images. Traditional approaches with hand-crafted visual descriptors might fail due to significant variations in image appearance. Recently, deep learning architectures provide promising solutions to address this problem using effective semantic representations. In this paper, we propose a deep convolutional hashing method that can be trained "point-wise" to simultaneously learn both semantic and binary representations of histopathological images. Specifically, we propose a convolutional neural network that introduces a latent binary encoding (LBE) layer for low-dimensional feature embedding to learn binary codes. We design a joint optimization objective function that encourages the network to learn discriminative representations from the label information, and reduce the gap between the real-valued low-dimensional embedded features and desired binary values. The binary encoding for new images can be obtained by forward propagating through the network and quantizing the output of the LBE layer. Experimental results on a large-scale histopathological image dataset demonstrate the effectiveness of the proposed method.

Neuroimaging Retrieval via Adaptive Ensemble Manifold Learning for Brain Disease Diagnosis

Alzheimer's disease (AD) is a neurodegenerative and non-curable disease, with serious cognitive impairment, such as dementia. Clinically, it is critical to study the disease with multi-source data in order to capture a global picture of it. In this respect, an adaptive ensemble manifold learning (AEML) algorithm is proposed to retrieve multi-source neuroimaging data. Specifically, an objective function based on manifold learning is formulated to impose geometrical constraints by similarity learning. The complementary characteristics of various sources of brain disease data for disorder discovery are investigated by tuning weights from ensemble learning. In addition, a generalized norm is explicitly explored for adaptive sparseness degree control. The proposed AEML algorithm is evaluated by the public AD neuroimaging initiative database. Results obtained from the extensive experiments demonstrate that our algorithm outperforms the traditional methods.

False-positive reduction in computer-aided mass detection using mammographic texture analysis and classification

BACKGROUND AND OBJECTIVE

The aim of computer-aided-detection (CAD) systems for mammograms is to assist radiologists by marking region of interest (ROIs) depicting abnormalities. However, the confusing appearance of some normal tissues that visually look like masses results in a large proportion of marked ROIs with normal tissues. This paper copes with this problem and proposes a framework to reduce false positive masses detected by CAD.

METHODS

To avoid the error induced by the segmentation step, we proposed a segmentation-free framework with particular attention to improve feature extraction and classification steps. We investigated for the first time in mammogram analysis, Hilbert's image representation, Kolmogorov-Smirnov distance and maximum subregion descriptors. Then, a feature selection step is performed to select the most discriminative features. Moreover, we considered several classifiers such as Random Forest, Support Vector Machine and Decision Tree to distinguish between normal tissues and masses. Our experiments were carried out on a large dataset of 10168 ROIs (8254 normal tissues and 1914 masses) constructed from the Digital Database for Screening Mammography (DDSM). To simulate practical scenario, our normal regions are false positives asserted by a CAD system from healthy cases.

RESULTS

The combination of all the descriptors yields better results than each feature set used alone, and the difference is statistically significant. Besides, the feature selection steps yields a statistically significant increase in the accuracy values for the three classifiers. Finally, the random forest achieves the highest accuracy (81.09%), outperforming the SVM classifier (80.01%)) and decision tree (79.12%), but the difference is not statistically significant.

CONCLUSIONS

The accuracy of discrimination between normal and abnormal ROIs in mammograms obtained with the proposed gray level texture features sets are encouraging and comparable to these obtained with multiresolution features. Combination of several features as well as feature selection steps improve the results. To improve false positives reduction in CAD systems for breast cancer diagnosis, these features could be combined with multiresolution features.

Prediction of radiographic abnormalities by the use of bag-of-features and convolutional neural networks

This study evaluated the feasibility of bag-of-features (BOF) and convolutional neural networks (CNN) for computer-aided detection in distinguishing normal from abnormal radiographic findings. Computed thoracic radiographs of dogs were collected. For the purposes of this study, radiographic findings were used to distinguish between normal and abnormal in the following areas: (1) normal cardiac silhouette vs. cardiomegaly, (2) normal lung vs. abnormal lung patterns, (3) normal mediastinal position vs. mediastinal shift, (4) normal pleural space vs. pleural effusion, and (5) normal pleural space vs. pneumothorax. Images for training and testing the models consisted of ventrodorsal and lateral projection images of the same scale. The number of images used for each finding are as follow: 3142 for cardiomegaly (1571 normal and 1571 abnormal from 1143 dogs), 2086 for lung pattern (1043 normal and 1043 abnormal from 1247 dogs), 892 for mediastinal shift (446 normal and 446 abnormal from 387 dogs), 940 for pleural effusion (470 normal and 470 abnormal from 284 dogs), and 78 for pneumothorax (39 normal and 39 abnormal from 61 dogs). All data samples were divided so that 60% would be used for training the algorithms and 40% for testing the two models. The performance of the classifiers was evaluated by calculating the accuracy, sensitivity and specificity. The accuracy of both models ranged from 79.6% to 96.9% in the testing set. CNN showed higher accuracy (CNN; 92.9-96.9% and BOF; 79.6-96.9%) and sensitivity (CNN; 92.1-100% and BOF; 74.1-94.8%) than BOF. In conclusion, both BOF and CNN have potential to be useful for improving work efficiency by double reading.

Overview on subjective similarity of images for content-based medical image retrieval

Computer-aided diagnosis systems for assisting the classification of various diseases have the potential to improve radiologists' diagnostic accuracy and efficiency, as reported in several studies. Conventional systems generally provide the probabilities of disease types in terms of numerical values, a method that may not be efficient for radiologists who are trained by reading a large number of images. Presentation of reference images similar to those of a new case being diagnosed can supplement the probability outputs based on computerized analysis as an intuitive guide, and it can assist radiologists in their diagnosis, reporting, and treatment planning. Many studies on content-based medical image retrievals have been reported on. For retrieval of perceptually similar and diagnostically relevant images, incorporation of perceptual similarity data by radiologists has been suggested. In this paper, studies on image retrieval methods are reviewed with a special focus on quantification, utilization, and the evaluation of subjective similarities between pairs of images.

Biomedical image classification based on a cascade of an SVM with a reject option and subspace analysis

Automated biomedical image classification could confront the challenges of high level noise, image blur, illumination variation and complicated geometric correspondence among various categorical biomedical patterns in practice. To handle these challenges, we propose a cascade method consisting of two stages for biomedical image classification. At stage 1, we propose a confidence score based classification rule with a reject option for a preliminary decision using the support vector machine (SVM). The testing images going through stage 1 are separated into two groups based on their confidence scores. Those testing images with sufficiently high confidence scores are classified at stage 1 while the others with low confidence scores are rejected and fed to stage 2. At stage 2, the rejected images from stage 1 are first processed by a subspace analysis technique called eigenfeature regularization and extraction (ERE), and then classified by another SVM trained in the transformed subspace learned by ERE. At both stages, images are represented based on two types of local features, i.e., SIFT and SURF, respectively. They are encoded using various bag-of-words (BoW) models to handle biomedical patterns with and without geometric correspondence, respectively. Extensive experiments are implemented to evaluate the proposed method on three benchmark real-world biomedical image datasets. The proposed method significantly outperforms several competing state-of-the-art methods in terms of classification accuracy.

Content-Based Image Retrieval System for Pulmonary Nodules: Assisting Radiologists in Self-Learning and Diagnosis of Lung Cancer

Visual information of similar nodules could assist the budding radiologists in self-learning. This paper presents a content-based image retrieval (CBIR) system for pulmonary nodules, observed in lung CT images. The reported CBIR systems of pulmonary nodules cannot be put into practice as radiologists need to draw the boundary of nodules during query formation and feature database creation. In the proposed retrieval system, the pulmonary nodules are segmented using a semi-automated technique, which requires a seed point on the nodule from the end-user. The involvement of radiologists in feature database creation is also reduced, as only a seed point is expected from radiologists instead of manual delineation of the boundary of the nodules. The performance of the retrieval system depends on the accuracy of the segmentation technique. Several 3D features are explored to improve the performance of the proposed retrieval system. A set of relevant shape and texture features are considered for efficient representation of the nodules in the feature space. The proposed CBIR system is evaluated for three configurations such as configuration-1 (composite rank of malignancy "1","2" as benign and "4","5" as malignant), configuration-2 (composite rank of malignancy "1","2", "3" as benign and "4","5" as malignant), and configuration-3 (composite rank of malignancy "1","2" as benign and "3","4","5" as malignant). Considering top 5 retrieved nodules and Euclidean distance metric, the precision achieved by the proposed method for configuration-1, configuration-2, and configuration-3 are 82.14, 75.91, and 74.27 %, respectively. The performance of the proposed CBIR system is close to the most recent technique, which is dependent on radiologists for manual segmentation of nodules. A computer-aided diagnosis (CAD) system is also developed based on CBIR paradigm. Performance of the proposed CBIR-based CAD system is close to performance of the CAD system using support vector machine.

Representing Medical Images With Encoded Local Projections

This paper introduces the "encoded local projections" (ELP) as a new dense-sampling image descriptor for search and classification problems. The gradient changes of multiple projections in local windows of gray-level images are encoded to build a histogram that captures spatial projection patterns. Using projections is a conventional technique in both medical imaging and computer vision. Furthermore, powerful dense-sampling methods, such as local binary patterns and the histogram of oriented gradients, are widely used for image classification and recognition. Inspired by many achievements of such existing descriptors, we explore the design of a new class of histogram-based descriptors with particular applications in medical imaging. We experiment with three public datasets (IRMA, Kimia Path24, and CT Emphysema) to comparatively evaluate the performance of ELP histograms. In light of the tremendous success of deep architectures, we also compare the results with deep features generated by pretrained networks. The results are quite encouraging as the ELP descriptor can surpass both conventional and deep descriptors in performance in several experimental settings.

Content-based image retrieval for Lung Nodule Classification Using Texture Features and Learned Distance Metric

Similarity measurement of lung nodules is a critical component in content-based image retrieval (CBIR), which can be useful in differentiating between benign and malignant lung nodules on computer tomography (CT). This paper proposes a new two-step CBIR scheme (TSCBIR) for computer-aided diagnosis of lung nodules. Two similarity metrics, semantic relevance and visual similarity, are introduced to measure the similarity of different nodules. The first step is to search for K most similar reference ROIs for each queried ROI with the semantic relevance metric. The second step is to weight each retrieved ROI based on its visual similarity to the queried ROI. The probability is computed to predict the likelihood of the queried ROI depicting a malignant lesion. In order to verify the feasibility of the proposed algorithm, a lung nodule dataset including 366 nodule regions of interest (ROIs) is assembled from LIDC-IDRI lung images on CT scans. Three groups of texture features are implemented to represent a nodule ROI. Our experimental results on the assembled lung nodule dataset show good performance improvement over existing popular classifiers.

Large-scale retrieval for medical image analytics: A comprehensive review

Over the past decades, medical image analytics was greatly facilitated by the explosion of digital imaging techniques, where huge amounts of medical images were produced with ever-increasing quality and diversity. However, conventional methods for analyzing medical images have achieved limited success, as they are not capable to tackle the huge amount of image data. In this paper, we review state-of-the-art approaches for large-scale medical image analysis, which are mainly based on recent advances in computer vision, machine learning and information retrieval. Specifically, we first present the general pipeline of large-scale retrieval, summarize the challenges/opportunities of medical image analytics on a large-scale. Then, we provide a comprehensive review of algorithms and techniques relevant to major processes in the pipeline, including feature representation, feature indexing, searching, etc. On the basis of existing work, we introduce the evaluation protocols and multiple applications of large-scale medical image retrieval, with a variety of exploratory and diagnostic scenarios. Finally, we discuss future directions of large-scale retrieval, which can further improve the performance of medical image analysis.

Feature Extraction and Classification on Esophageal X-Ray Images of Xinjiang Kazak Nationality

Esophageal cancer is one of the fastest rising types of cancers in China. The Kazak nationality is the highest-risk group in Xinjiang. In this work, an effective computer-aided diagnostic system is developed to assist physicians in interpreting digital X-ray image features and improving the quality of diagnosis. The modules of the proposed system include image preprocessing, feature extraction, feature selection, image classification, and performance evaluation. 300 original esophageal X-ray images were resized to a region of interest and then enhanced by the median filter and histogram equalization method. 37 features from textural, frequency, and complexity domains were extracted. Both sequential forward selection and principal component analysis methods were employed to select the discriminative features for classification. Then, support vector machine and K-nearest neighbors were applied to classify the esophageal cancer images with respect to their specific types. The classification performance was evaluated in terms of the area under the receiver operating characteristic curve, accuracy, precision, and recall, respectively. Experimental results show that the classification performance of the proposed system outperforms the conventional visual inspection approaches in terms of diagnostic quality and processing time. Therefore, the proposed computer-aided diagnostic system is promising for the diagnostics of esophageal cancer.

Scalable histopathological image analysis via supervised hashing with multiple features

Histopathology is crucial to diagnosis of cancer, yet its interpretation is tedious and challenging. To facilitate this procedure, content-based image retrieval methods have been developed as case-based reasoning tools. Especially, with the rapid growth of digital histopathology, hashing-based retrieval approaches are gaining popularity due to their exceptional efficiency and scalability. Nevertheless, few hashing-based histopathological image analysis methods perform feature fusion, despite the fact that it is a common practice to improve image retrieval performance. In response, we exploit joint kernel-based supervised hashing (JKSH) to integrate complementary features in a hashing framework. Specifically, hashing functions are designed based on linearly combined kernel functions associated with individual features. Supervised information is incorporated to bridge the semantic gap between low-level features and high-level diagnosis. An alternating optimization method is utilized to learn the kernel combination and hashing functions. The obtained hashing functions compress multiple high-dimensional features into tens of binary bits, enabling fast retrieval from a large database. Our approach is extensively validated on 3121 breast-tissue histopathological images by distinguishing between actionable and benign cases. It achieves 88.1% retrieval precision and 91.3% classification accuracy within 16.5 ms query time, comparing favorably with traditional methods.

Retrieval of Brain Tumors by Adaptive Spatial Pooling and Fisher Vector Representation

Content-based image retrieval (CBIR) techniques have currently gained increasing popularity in the medical field because they can use numerous and valuable archived images to support clinical decisions. In this paper, we concentrate on developing a CBIR system for retrieving brain tumors in T1-weighted contrast-enhanced MRI images. Specifically, when the user roughly outlines the tumor region of a query image, brain tumor images in the database of the same pathological type are expected to be returned. We propose a novel feature extraction framework to improve the retrieval performance. The proposed framework consists of three steps. First, we augment the tumor region and use the augmented tumor region as the region of interest to incorporate informative contextual information. Second, the augmented tumor region is split into subregions by an adaptive spatial division method based on intensity orders; within each subregion, we extract raw image patches as local features. Third, we apply the Fisher kernel framework to aggregate the local features of each subregion into a respective single vector representation and concatenate these per-subregion vector representations to obtain an image-level signature. After feature extraction, a closed-form metric learning algorithm is applied to measure the similarity between the query image and database images. Extensive experiments are conducted on a large dataset of 3604 images with three types of brain tumors, namely, meningiomas, gliomas, and pituitary tumors. The mean average precision can reach 94.68%. Experimental results demonstrate the power of the proposed algorithm against some related state-of-the-art methods on the same dataset.

Multi-View Probabilistic Classification of Breast Microcalcifications

Classification of clustered breast microcalcifications into benign and malignant categories is an extremely challenging task for computerized algorithms and expert radiologists alike. In this paper we apply a multi-view-classifier for the task. We describe a two-step classification method that is based on a view-level decision, implemented by a logistic regression classifier, followed by a stochastic combination of the two view-level indications into a single benign or malignant decision. The proposed method was evaluated on a large number of cases from a standardized digital database for screening mammography (DDSM). Experimental results demonstrate the advantage of the proposed multi-view classification algorithm that automatically learns the best way to combine the views.

Dictionary Pruning with Visual Word Significance for Medical Image Retrieval

Content-based medical image retrieval (CBMIR) is an active research area for disease diagnosis and treatment but it can be problematic given the small visual variations between anatomical structures. We propose a retrieval method based on a bag-of-visual-words (BoVW) to identify discriminative characteristics between different medical images with Pruned Dictionary based on Latent Semantic Topic description. We refer to this as the PD-LST retrieval. Our method has two main components. First, we calculate a topic-word significance value for each visual word given a certain latent topic to evaluate how the word is connected to this latent topic. The latent topics are learnt, based on the relationship between the images and words, and are employed to bridge the gap between low-level visual features and high-level semantics. These latent topics describe the images and words semantically and can thus facilitate more meaningful comparisons between the words. Second, we compute an overall-word significance value to evaluate the significance of a visual word within the entire dictionary. We designed an iterative ranking method to measure overall-word significance by considering the relationship between all latent topics and words. The words with higher values are considered meaningful with more significant discriminative power in differentiating medical images. We evaluated our method on two public medical imaging datasets and it showed improved retrieval accuracy and efficiency.

Breast cancer diagnosis in digitized mammograms using curvelet moments

BACKGROUND

Feature extraction is a key issue in designing a computer aided diagnosis system. Recent researches on breast cancer diagnosis have reported the effectiveness of multiscale transforms (wavelets and curvelets) for mammogram analysis and have shown the superiority of curvelet transform. However, the curse of dimensionality problem arises when using the curvelet coefficients and therefore a reduction method is required to extract a reduced set of discriminative features.

METHODS

This paper deals with this problem and proposes a feature extraction method based on curvelet transform and moment theory for mammogram description. First, we performed discrete curvelet transform and we computed the four first-order moments from curvelet coefficients distribution. Hence, two feature sets can be obtained: moments from each band and moments from each level. In this work, both sets are studied. Then, the t-test ranking technique was applied to select the best features from each set. Finally, a k-nearest neighbor classifier was used to distinguish between normal and abnormal breast tissues and to classify tumors as malignant or benign. Experiments were performed on 252 mammograms from the Mammographic Image Analysis Society (mini-MIAS) database using the leave-one-out cross validation as well as on 11553 mammograms from the Digital Database for Screening Mammography (DDSM) database using 2×5-fold cross validation.

RESULTS

Experimental results prove the effectiveness and the superiority of curvelet moments for mammogram analysis. Indeed, results on the mini-MIAS database show that curvelet moments yield an accuracy of 91.27% (resp. 81.35 %) with 10 (resp. 8) features for abnormality (resp. malignancy) detection. In addition, empirical comparisons of the proposed method against state-of-the-art curvelet-based methods on the DDSM database show that the suggested method does not only lead to a more reduced feature set, but it also statistically outperforms all the compared methods in terms of accuracy.

CONCLUSIONS

In summary, curvelet moments are an efficient and effective way to extract a reduced set of discriminative features for breast cancer diagnosis.

Measuring sparse temporal-variation for accurate registration of dynamic contrast-enhanced breast MR images

Accurate registration of dynamic contrast-enhanced (DCE) MR breast images is challenging due to the temporal variations of image intensity and the non-rigidity of breast motion. The former can cause the well-known tumor shrinking/expanding problem in registration process while the latter complicates the task by requiring an estimation of non-rigid deformation. In this paper, we treat the intensity's temporal variations as "corruptions" which spatially distribute in a sparse pattern and model them with a L1 norm and a Lorentzian norm. We show that these new image similarity measurements can characterize the non-Gaussian property of the difference between the pre-contrast and post-contrast images and help to resolve the shrinking/expanding problem by forgiving significant image variations. Furthermore, we propose an iteratively re-weighted least squares based method and a linear programming based technique for optimizing the objective functions obtained using these two novel norms. We show that these optimization techniques outperform the traditional gradient-descent approach. Experimental results with sequential DCE-MR images from 28 patients show the superior performances of our algorithms.