Texture classification and retrieval using shearlets and linear regression

Early Detection of Medical Image Analysis by Using Machine Learning Method

We develop effective medical image classification techniques, with an emphasis on histopathology and magnetic resonance imaging (MRI). The trainer utilized the curriculum as a starting point for a set of data and a restricted number of samples, and we used it as a starting point for a set of data. As calibrating a machine learning model is difficult, we used alternative methods as unsupervised feature extracts or weight-conditioning factors for identifying pathological histology pictures. As a result, the pretrained models will be trained on 3-channel RGB pictures, while the MRI sample has more slices. To alter the working model using the MRI data, the convolutional neural network (CNN) must be fine-tuned. Pretrained models are placed and then used as feature snippets. However, there is a scarcity of well-done medical photos, making training machine learning models a difficult endeavor to begin with. In any case, data augmentation aids in the generation of sufficient training samples; however, it is unclear if data augmentation aids in the prediction of unknown data samples. As a result, we fine-tuned machine learning models without using any additional data. Furthermore, rather than utilizing a standard machine learning classifier for the MRI data, we created a unique CNN that uses both 3D shear descriptors and deep features as input. This custom network identifies the MRI sample after processing our representation of the characteristics from beginning to end. On the hidden MRI dataset, our bespoke CNN outperforms traditional machine learning. Our CNN model is less prone to overfitting as a result of this. Furthermore, we have given cutting-edge outcomes employing machine learning.

Local bit-plane neighbour dissimilarity pattern in non-subsampled shearlet transform domain for bio-medical image retrieval

This paper introduces a novel descriptor non-subsampled shearlet transform (NSST) local bit-plane neighbour dissimilarity pattern (NSST-LBNDP) for biomedical image retrieval based on NSST, bit-plane slicing and local pattern based features. In NSST-LBNDP, the input image is first decomposed by NSST, followed by introduction of non-linearity on the NSST coefficients by computing local energy features. The local energy features are next normalized into 8-bit values. The multiscale NSST is used to provide translational invariance and has flexible directional sensitivity to catch more anisotropic information of an image. The normalised NSST subband features are next decomposed into bit-plane slices in order to capture very fine to coarse subband details. Then each bit-plane slices of all the subbands are encoded by exploiting the dissimilarity relationship between each neighbouring pixel and its adjacent neighbours. Experiments on two computed tomography (CT) and one magnetic resonance imaging (MRI) image datasets confirms the superior results of NSST-LBNDP when compared to many recent well known relevant descriptors both in terms of average retrieval precision (ARP) and average retrieval recall (ARR).

Intelligent vision based wear forecasting on surfaces of machine tool elements

Abstract

To realize autonomous production machines it is necessary that machines are able to automatically and autonomously predict their condition. Although many classical as well as Deep Learning based approaches have shown the ability to classify faults, so far there are no approaches that go beyond the basic detection of faults. A complete, image based predictive maintenance approach for machine tool components has to the best of our knowledge not been investigated so far. In this paper it is shown how defects on a Ball Screw Drive (BSD) can be automatically detected by using a machine learning based detection module, quantified by using an intelligent defect quantification module and finally forecasted by a prognosis module in a combined approach. To optimize the presented method, it is shown how existing domain knowledge can be formalized in an expert system and combined with the predictions of the machine learning model to aid quality of the prediction system. This enables the practitioner to forecast the severity of failures on BSD and prevent machine breakdowns. The work is intended to set new accents for the development of practical predictive maintenance systems and bridging the fields of machine learning and production engineering. The code is available under: https://github.com/2Obe/Pitting_Pred_Maintenance.

YOT-Net: YOLOv3 Combined Triplet Loss Network for Copper Elbow Surface Defect Detection

Copper elbows are an important product in industry. They are used to connect pipes for transferring gas, oil, and liquids. Defective copper elbows can lead to serious industrial accidents. In this paper, a novel model named YOT-Net (YOLOv3 combined triplet loss network) is proposed to automatically detect defective copper elbows. To increase the defect detection accuracy, triplet loss function is employed in YOT-Net. The triplet loss function is introduced into the loss module of YOT-Net, which utilizes image similarity to enhance feature extraction ability. The proposed method of YOT-Net shows outstanding performance in copper elbow surface defect detection.

3D shearlet-based descriptors combined with deep features for the classification of Alzheimer's disease based on MRI data

Alzheimer's disease (AD) is a neurodegenerative disease that afflicts millions of people worldwide. Early detection of AD is critical, as drug trials show a promising advantage to those patients with early diagnoses. In this study, magnetic resonance imaging (MRI) datasets from the Alzheimer's Disease Neuroimaging Initiative (ADNI) and The Open Access Series of Imaging Studies are used. Our method for performing the classification of AD is to combine a set of shearlet-based descriptors with deep features. A major challenge in classifying such MRI datasets is the high dimensionality of feature vectors because of the large number of slices of each MRI sample. Given the volumetric nature of the MRI data, we propose using the 3D shearlet transform (3D-ST), but we obtain the average of all directionalities, which reduces the dimensionality. On the other hand, we propose to leverage the capabilities of convolutional neural networks (CNN) to learn feature maps from stacked MRI slices, which generate a very compact feature vector for each MRI sample. The 3D-ST and CNN feature vectors are combined for the classification of AD. After the concatenation of the feature vectors, they are used to train a classifier. Alternatively, a custom CNN model is utilized, in which the descriptors are further processed end to end to obtain the classification model. Our experimental results show that the fusion of shearlet-based descriptors and deep features improves classification performance, especially on the ADNI dataset.

Texture Analysis and its Applications in Biomedical Imaging: A Survey

Texture analysis describes a variety of image analysis techniques that quantify the variation in intensity and pattern. This paper provides an overview of several texture analysis approaches addressing the rationale supporting them, their advantages, drawbacks, and applications. This surveys emphasis is in collecting and categorising over five decades of active research on texture analysis. Brief descriptions of different approaches are presented along with application examples. From a broad range of texture analysis applications, this surveys final focus is on biomedical image analysis. An up-to-date list of biological tissues and organs in which disorders produce texture changes that may be used to spot disease onset and progression is provided. Finally, the role of texture analysis methods as biomarkers of disease is summarised.

Group-Wise Learning for Aurora Image Classification With Multiple Representations

In conventional aurora image classification methods, it is general to employ only one single feature representation to capture the morphological characteristics of aurora images, which is difficult to describe the complicated morphologies of different aurora categories. Although several studies have proposed to use multiple feature representations, the inherent correlation among these representations are usually neglected. To address this problem, we propose a group-wise learning (GWL) method for the automatic aurora image classification using multiple representations. Specifically, we first extract the multiple feature representations for aurora images, and then construct a graph in each of multiple feature spaces. To model the correlation among different representations, we partition multiple graphs into several groups via a clustering algorithm. We further propose a GWL model to automatically estimate class labels for aurora images and optimal weights for the multiple representations in a data-driven manner. Finally, we develop a label fusion approach to make a final classification decision for new testing samples. The proposed GWL method focuses on the diverse properties of multiple feature representations, by clustering the correlated representations into the same group. We evaluate our method on an aurora image data set that contains 12 682 aurora images from 19 days. The experimental results demonstrate that the proposed GWL method achieves approximately 6% improvement in terms of classification accuracy, compared to the methods using a single feature representation.

C-CNN: Contourlet Convolutional Neural Networks

Extracting effective features is always a challenging problem for texture classification because of the uncertainty of scales and the clutter of textural patterns. For texture classification, spectral analysis is traditionally employed in the frequency domain. Recent studies have shown the potential of convolutional neural networks (CNNs) when dealing with the texture classification task in the spatial domain. In this article, we try combining both approaches in different domains for more abundant information and proposed a novel network architecture named contourlet CNN (C-CNN). The network aims to learn sparse and effective feature representations for images. First, the contourlet transform is applied to get the spectral features from an image. Second, the spatial-spectral feature fusion strategy is designed to incorporate the spectral features into CNN architecture. Third, the statistical features are integrated into the network by the statistical feature fusion. Finally, the results are obtained by classifying the fusion features. We also investigated the behavior of the parameters in contourlet decomposition. Experiments on the widely used three texture data sets (kth-tips2-b, DTD, and CUReT) and five remote sensing data sets (UCM, WHU-RS, AID, RSSCN7, and NWPU-RESISC45) demonstrate that the proposed approach outperforms several well-known classification methods in terms of classification accuracy with fewer trainable parameters.

Texture features in the Shearlet domain for histopathological image classification

Background

A various number of imaging modalities are available (e.g., magnetic resonance, x-ray, ultrasound, and biopsy) where each modality can reveal different structural aspects of tissues. However, the analysis of histological slide images that are captured using a biopsy is considered the gold standard to determine whether cancer exists. Furthermore, it can reveal the stage of cancer. Therefore, supervised machine learning can be used to classify histopathological tissues. Several computational techniques have been proposed to study histopathological images with varying levels of success. Often handcrafted techniques based on texture analysis are proposed to classify histopathological tissues which can be used with supervised machine learning.

Methods

In this paper, we construct a novel feature space to automate the classification of tissues in histology images. Our feature representation is to integrate various features sets into a new texture feature representation. All of our descriptors are computed in the complex Shearlet domain. With complex coefficients, we investigate not only the use of magnitude coefficients, but also study the effectiveness of incorporating the relative phase (RP) coefficients to create the input feature vector. In our study, four texture-based descriptors are extracted from the Shearlet coefficients: co-occurrence texture features, Local Binary Patterns, Local Oriented Statistic Information Booster, and segmentation-based Fractal Texture Analysis. Each set of these attributes captures significant local and global statistics. Therefore, we study them individually, but additionally integrate them to boost the accuracy of classifying the histopathology tissues while being fed to classical classifiers. To tackle the problem of high-dimensionality, our proposed feature space is reduced using principal component analysis. In our study, we use two classifiers to indicate the success of our proposed feature representation: Support Vector Machine (SVM) and Decision Tree Bagger (DTB).

Results

Our feature representation delivered high performance when used on four public datasets. As such, the best achieved accuracy: multi-class Kather (i.e., 92.56%), BreakHis (i.e., 91.73%), Epistroma (i.e., 98.04%), Warwick-QU (i.e., 96.29%).

Conclusions

Our proposed method in the Shearlet domain for the classification of histopathological images proved to be effective when it was investigated on four different datasets that exhibit different levels of complexity.

Hierarchical Feature Fusion Network for Salient Object Detection

Convolutional Neural Network (CNN) has shown their advantages in salient object detection. CNN can generate great saliency maps because it can obtain high-level semantic information. And the semantic information is usually achieved by stacking multiple convolutional layers and pooling layers. However, multiple pooling operations will reduce the size of the feature map and easily blur the boundary of the salient object. Therefore, such operations are not beneficial to generate great saliency results. To alleviate this issue, we propose a novel edge information-guided hierarchical feature fusion network (HFFNet). Our network fuses features hierarchically and retains accurate semantic information and clear edge information effectively. Specifically, we extract image features from different levels of VGG. Then, we fuse the features hierarchically to generate high-level semantic information and low-level edge information. In order to retain better information at different levels, we adopt a one-to-one hierarchical supervision strategy to supervise the generation of low-level information and high-level information respectively. Finally, we use low-level edge information to guide the saliency map generation, and the edge guidance fusion is able to identify saliency regions effectively. The proposed HFFNet has been extensively evaluated on five traditional benchmark datasets. The experimental results demonstrate that the proposed model is fairly effective in salient object detection compared with 10 state-of-the-art models under different evaluation indicators, and it is superior to most of the comparison models.

Deep Multi-scale Discriminative Networks for Double JPEG Compression Forensics

As JPEG is the most widely used image format, the importance of tampering detection for JPEG images in blind forensics is self-evident. In this area, extracting effective statistical characteristics from a JPEG image for classification remains a challenge. Effective features are designed manually in traditional methods, suggesting that extensive labor-consuming research and derivation is required. In this article, we propose a novel image tampering detection method based on deep multi-scale discriminative networks (MSD-Nets). The multi-scale module is designed to automatically extract multiple features from the discrete cosine transform (DCT) coefficient histograms of the JPEG image. This module can capture the characteristic information in different scale spaces. In addition, a discriminative module is also utilized to improve the detection effect of the networks in those difficult situations when the first compression quality ( QF 1) is higher than the second one ( QF 2). A special network in this module is designed to distinguish the small statistical difference between authentic and tampered regions in these cases. Finally, a probability map can be obtained and the specific tampering area is located using the last classification results. Extensive experiments demonstrate the superiority of our proposed method in both quantitative and qualitative metrics when compared with state-of-the-art approaches.

Discriminative and Orthogonal Subspace Constraints-Based Nonnegative Matrix Factorization

Nonnegative matrix factorization (NMF) is one widely used feature extraction technology in the tasks of image clustering and image classification. For the former task, various unsupervised NMF methods based on the data distribution structure information have been proposed. While for the latter task, the label information of the dataset is one very important guiding. However, most previous proposed supervised NMF methods emphasis on imposing the discriminant constraints on the coefficient matrix. When dealing with new coming samples, the transpose or the pseudoinverse of the basis matrix is used to project these samples to the low dimension space. In this way, the label influence to the basis matrix is indirect. Although, there are also some methods trying to constrain the basis matrix in NMF framework, either they only restrict within-class samples or impose improper constraint on the basis matrix. To address these problems, in this article a novel NMF framework named discriminative and orthogonal subspace constraints-based nonnegative matrix factorization (DOSNMF) is proposed. In DOSNMF, the discriminative constraints are imposed on the projected subspace instead of the directly learned representation. In this manner, the discriminative information is directly connected with the projected subspace. At the same time, an orthogonal term is incorporated in DOSNMF to adjust the orthogonality of the learned basis matrix, which can ensure the orthogonality of the learned subspace and improve the sparseness of the basis matrix at the same time. This framework can be implemented in two ways. The first way is based on the manifold learning theory. In this way, two graphs, i.e., the intrinsic graph and the penalty graph, are constructed to capture the intra-class structure and the inter-class distinctness. With this design, both the manifold structure information and the discriminative information of the dataset are utilized. For convenience, we name this method as the name of the framework, i.e., DOSNMF. The second way is based on the Fisher’s criterion, we name it Fisher’s criterion-based DOSNMF (FDOSNMF). The objective functions of DOSNMF and FDOSNMF can be easily optimized using multiplicative update (MU) rules. The new methods are tested on five datasets and compared with several supervised and unsupervised variants of NMF. The experimental results reveal the effectiveness of the proposed methods.

Training-Based Gradient LBP Feature Models for Multiresolution Texture Classification

Local binary pattern (LBP) is a simple, yet efficient coding model for extracting texture features. To improve texture classification, this paper designs a median sampling regulation, defines a group of gradient LBP (gLBP) descriptors, proposes a training-based feature model mapping method, and then develops a texture classification frame using the multiresolution feature fusion of four gLBP descriptors. Cooperated by median sampling, four descriptors encode a pixel respectively by central gradient, radial gradient, magnitude gradient and tangent gradient to generate initial gLBP patterns. The feature mapping models of gLBP descriptors are constructed by the maximal relative-variation rate (mr2) of rotation-invariant patterns, and then prestored as mapping lookup files. By mapping, initial patterns can be transformed into low-dimensional ones. And then it generates multiresolution texture features via the joint and concatenation of gLBP descriptors on different sampling parameters. A trained nearest neighbor classifier with chi-square distance is applied to classify textures by feature histograms. The experimental results of simulation on five public texture databases show that the proposed method is reliable and efficient in texture classification. In comparison with nine other similar approaches, including two state-of-the-art ones, the proposed method runs faster than most of them and also outperforms all of them in terms of classification accuracy and noise robustness. It achieves higher accuracy and has also better robustness to the Salt&Pepper and Gaussian noise added artificially into texture images.

A 3-D Gabor Phase-Based Coding and Matching Framework for Hyperspectral Imagery Classification

As manual labeling is very difficult and time-consuming, the labeled samples used to train a supervised classifier are generally limited, which become one of the biggest challenge for hyperspectral imagery classification. In order to tackle this issue, a recent trend is to exploit the structure information of materials, as which reflects the region homogeneity in the spatial domain and offers an invaluable complement to the spectral information. In this respect, 3-D Gabor wavelets have been introduced to extract joint spectral-spatial features for hyperspectral images. One the one hand, the features extracted by 3-D Gabor wavelets lead to very good performance for classification. On the other hand, its drawbacks, i.e., big number of features and high computational cost, limit its applicability. In this paper, a 3-D Gabor-wavelet-based phase coding and Hamming distance-based matching (3DGPC-HDM) framework is developed for hyperspectral imagery classification. The proposed method, instead of taking into account the large volume of Gabor magnitude features, exploits the Gabor phase features with certain orientations (i.e., the direction parallel to the spectral axis), which are then encoded by a simple quadrant bit coding scheme. After that, a normalized Hamming distance matching (HDM) method is adopted to determine the similarity of two samples, and the nearest neighbor classifier is routinely utilized for pixelwise recognition. Finally, experiments on three real hyperspectral data sets show that the proposed 3DGPC-HDM leads to very good performance. Comparisons with the state-of-the-art methods in the literature, in terms of both classifier complexity and generalization ability from very small training sets, are also included.

Graph Regularized Non-Negative Low-Rank Matrix Factorization for Image Clustering

Non-negative matrix factorization (NMF) has been one of the most popular methods for feature learning in the field of machine learning and computer vision. Most existing works directly apply NMF on high-dimensional image datasets for computing the effective representation of the raw images. However, in fact, the common essential information of a given class of images is hidden in their low rank parts. For obtaining an effective low-rank data representation, we in this paper propose a non-negative low-rank matrix factorization (NLMF) method for image clustering. For the purpose of improving its robustness for the data in a manifold structure, we further propose a graph regularized NLMF by incorporating the manifold structure information into our proposed objective function. Finally, we develop an efficient alternating iterative algorithm to learn the low-dimensional representation of low-rank parts of images for clustering. Alternatively, we also incorporate robust principal component analysis into our proposed scheme. Experimental results on four image datasets reveal that our proposed methods outperform four representative methods.

Visual Classification of Furniture Styles

Furniture style describes the discriminative appearance characteristics of furniture. It plays an important role in real-world indoor decoration. In this article, we explore the furniture style features and study the problem of furniture style classification. Differing from traditional object classification, furniture style classification aims at classifying different furniture in terms of the “style” that describes its appearance (e.g., American style, Gothic style, Rococo style, etc.) rather than the “kind” that is more related to its functional structure (e.g., bed, desk, etc.). To pursue efficient furniture style features, we construct a novel dataset of furniture styles that contains 16 common style categories and implement three strategies with respect to two categories of classification, that is, handcrafted classification and learning-based classification. First, we follow the typical image classification pipeline to extract the handcrafted features and train the classifier by support vector machine. Then we use the convolutional neural network to extract learning-based features from training images. To obtain comprehensive furniture style features, we finally combine the handcrafted image classification pipeline and the learning-based network. We experimentally evaluate the performances of handcrafted features and learning-based features of each strategy, and the results show the superiority of learning-based features and also the comprehensiveness of handcrafted features.

Learning Instance Correlation Functions for Multilabel Classification

Multilabel learning has a wide range of potential applications in reality. It attracts a great deal of attention during the past years and has been extensively studied in many fields including image annotation and text categorization. Although many efforts have been made for multilabel learning, there are two challenging issues remaining, i.e., how to exploit the correlations and how to tackle the high-dimensional problems of multilabel data. In this paper, an effective algorithm is developed for multilabel classification with utilizing those data that are relevant to the targets. The key is the construction of a coefficient-based mapping between training and test instances, where the mapping relationship exploits the correlations among the instances, rather than the explicit relationship between the variables and the class labels of data. Further, a constraint, ℓ¹-norm penalty, is performed on the mapping relationship to make the model sparse, weakening the impacts of noisy data. Our empirical study on eight public datasets shows that the proposed method is more effective in comparing with the state-of-the-art multilabel classifiers.

Exploring Deep Learning and Transfer Learning for Colonic Polyp Classification

Recently, Deep Learning, especially through Convolutional Neural Networks (CNNs) has been widely used to enable the extraction of highly representative features. This is done among the network layers by filtering, selecting, and using these features in the last fully connected layers for pattern classification. However, CNN training for automated endoscopic image classification still provides a challenge due to the lack of large and publicly available annotated databases. In this work we explore Deep Learning for the automated classification of colonic polyps using different configurations for training CNNs from scratch (or full training) and distinct architectures of pretrained CNNs tested on 8-HD-endoscopic image databases acquired using different modalities. We compare our results with some commonly used features for colonic polyp classification and the good results suggest that features learned by CNNs trained from scratch and the “off-the-shelf” CNNs features can be highly relevant for automated classification of colonic polyps. Moreover, we also show that the combination of classical features and “off-the-shelf” CNNs features can be a good approach to further improve the results.

Directional wavelet based features for colonic polyp classification

In this work, various wavelet based methods like the discrete wavelet transform, the dual-tree complex wavelet transform, the Gabor wavelet transform, curvelets, contourlets and shearlets are applied for the automated classification of colonic polyps. The methods are tested on 8 HD-endoscopic image databases, where each database is acquired using different imaging modalities (Pentax's i-Scan technology combined with or without staining the mucosa), 2 NBI high-magnification databases and one database with chromoscopy high-magnification images. To evaluate the suitability of the wavelet based methods with respect to the classification of colonic polyps, the classification performances of 3 wavelet transforms and the more recent curvelets, contourlets and shearlets are compared using a common framework. Wavelet transforms were already often and successfully applied to the classification of colonic polyps, whereas curvelets, contourlets and shearlets have not been used for this purpose so far. We apply different feature extraction techniques to extract the information of the subbands of the wavelet based methods. Most of the in total 25 approaches were already published in different texture classification contexts. Thus, the aim is also to assess and compare their classification performance using a common framework. Three of the 25 approaches are novel. These three approaches extract Weibull features from the subbands of curvelets, contourlets and shearlets. Additionally, 5 state-of-the-art non wavelet based methods are applied to our databases so that we can compare their results with those of the wavelet based methods. It turned out that extracting Weibull distribution parameters from the subband coefficients generally leads to high classification results, especially for the dual-tree complex wavelet transform, the Gabor wavelet transform and the Shearlet transform. These three wavelet based transforms in combination with Weibull features even outperform the state-of-the-art methods on most of the databases. We will also show that the Weibull distribution is better suited to model the subband coefficient distribution than other commonly used probability distributions like the Gaussian distribution and the generalized Gaussian distribution. So this work gives a reasonable summary of wavelet based methods for colonic polyp classification and the huge amount of endoscopic polyp databases used for our experiments assures a high significance of the achieved results.

On Visual Texture Preference: Can an Ecological Model Explain Why People Like Some Textures More Than Others?

What visual textures do people like and why? Here, we test whether the ecological valence theory proposed for color preferences can also predict people’s preferences for visual texture. According to the theory, people should like visual textures associated with positive objects or entities and dislike visual textures associated with negative objects or entities. We compare the results for the ecological model with a more traditional texture-preference model based on computational features and find that the ecological model performs reasonably well considering its lower complexity, explaining 63% of the variance in the human preference data.