Document classification using convolutional neural networks with small window sizes and latent semantic analysis

A parsimonious convolutional neural network (CNN) for text document classification that replicates the ease of use and high classification performance of linear methods is presented. This new CNN architecture can leverage locally trained latent semantic analysis (LSA) word vectors. The architecture is based on parallel 1D convolutional layers with small window sizes, ranging from 1 to 5 words. To test the efficacy of the new CNN architecture, three balanced text datasets that are known to perform exceedingly well with linear classifiers were evaluated. Also, three additional imbalanced datasets were evaluated to gauge the robustness of the LSA vectors and small window sizes. The new CNN architecture consisting of 1 to 4-grams, coupled with LSA word vectors, exceeded the accuracy of all linear classifiers on balanced datasets with an average improvement of 0.73%. In four out of the total six datasets, the LSA word vectors provided a maximum classification performance on par with or better than word2vec vectors in CNNs. Furthermore, in four out of the six datasets, the new CNN architecture provided the highest classification performance. Thus, the new CNN architecture and LSA word vectors could be used as a baseline method for text classification tasks.

Improving clustering performance using independent component analysis and unsupervised feature learning

Objective

To provide a parsimonious clustering pipeline that provides comparable performance to deep learning-based clustering methods, but without using deep learning algorithms, such as autoencoders.

Materials and methods

Clustering was performed on six benchmark datasets, consisting of five image datasets used in object, face, digit recognition tasks (COIL20, COIL100, CMU-PIE, USPS, and MNIST) and one text document dataset (REUTERS-10K) used in topic recognition. K-means, spectral clustering, Graph Regularized Non-negative Matrix Factorization, and K-means with principal components analysis algorithms were used for clustering. For each clustering algorithm, blind source separation (BSS) using Independent Component Analysis (ICA) was applied. Unsupervised feature learning (UFL) using reconstruction cost ICA (RICA) and sparse filtering (SFT) was also performed for feature extraction prior to the cluster algorithms. Clustering performance was assessed using the normalized mutual information and unsupervised clustering accuracy metrics.

Results

Performing, ICA BSS after the initial matrix factorization step provided the maximum clustering performance in four out of six datasets (COIL100, CMU-PIE, MNIST, and REUTERS-10K). Applying UFL as an initial processing component helped to provide the maximum performance in three out of six datasets (USPS, COIL20, and COIL100). Compared to state-of-the-art non-deep learning clustering methods, ICA BSS and/or UFL with graph-based clustering algorithms outperformed all other methods. With respect to deep learning-based clustering algorithms, the new methodology presented here obtained the following rankings: COIL20, 2nd out of 5; COIL100, 2nd out of 5; CMU-PIE, 2nd out of 5; USPS, 3rd out of 9; MNIST, 8th out of 15; and REUTERS-10K, 4th out of 5.

Discussion

By using only ICA BSS and UFL using RICA and SFT, clustering accuracy that is better or on par with many deep learning-based clustering algorithms was achieved. For instance, by applying ICA BSS to spectral clustering on the MNIST dataset, we obtained an accuracy of 0.882. This is better than the well-known Deep Embedded Clustering algorithm that had obtained an accuracy of 0.818 using stacked denoising autoencoders in its model.

Conclusion

Using the new clustering pipeline presented here, effective clustering performance can be obtained without employing deep clustering algorithms and their accompanying hyper-parameter tuning procedure.

Predicting and Grouping Digitized Paintings by Style using Unsupervised Feature Learning

OBJECTIVE

To create an system to aid in the analysis of art history by classifying and grouping digitized paintings based on stylistic features automatically learned without prior knowledge.

MATERIAL AND METHODS

6,776 digitized paintings from 8 different artistic styles (Art Nouveau, Baroque, Expressionism, Impressionism, Realism, Romanticism, Renaissance, and Post-Impressionism) were utilized to classify (predict) and cluster (group) paintings according to style. The method of unsupervised feature learning with K-means (UFLK), inspired by deep learning, was utilized to extract features from the paintings. These features were then used in: (1) a support vector machine algorithm to classify the style of new test paintings based on a training set of paintings having known style labels; and (2) a spectral clustering algorithm to group the paintings into distinct style groups (anonymously, without employing any known style labels). Classification performance was determined by accuracy and F-score. Clustering performance was determined by: 1) the ability to recover the original stylistic groupings (using a cost analysis of all possible combinations of 8 group label assignments); 2) F-score; and 3) a reliability analysis. The latter analysis used two novel ways to determine the distribution of the null-hypothesis: (a) a uniform distribution projected onto the principal components of the original data; and (b) a randomized, weighted adjacency matrix. The ability to gain insights into art was tested by a semantic analysis of the clustering results. For this purpose, we represented the featural characteristics of each painting by an N-dimensional feature vector, and plotted the distance between vector endpoints (i.e., similarity between paintings). Then, we color-coded the endpoints with the assigned lowest-cost style labels. The scatterplot was visually inspected for separation of the paintings, where the amount of separation between color clusters provides semantic information on the interrelatedness between styles.

RESULTS

The UFLK-extracted features resembled the edges/lines/colors in the paintings. For feature-based classification of paintings, the macro-averaged F-score was 0.469. Classification accuracy and F-score were similar/higher compared to other classification methods using more complex feature learning models (e.g., convolutional neural networks, a supervised algorithm). The clustering via UFLK-extracted features yielded 8 unlabeled style groupings. In 6 of 8 clusters, the most common true painting style matched the cluster style assigned by cost analysis. The clustering had an F-score of 0.212. (There are no comparison methods for clustering paintings.) For the semantic analysis, the featural characteristics of Baroque and Art Nouveau were found to be similar, indicating a relationship between these styles.

DISCUSSION/CONCLUSION

The UFLK method can extract features from digitized paintings. We were able to extract characteristics of art without any prior information about the nature of the features or the stylistic designation of the paintings. The methods herein may provide art researchers with the latest computational techniques for the documentation, interpretation, and forensics of art. The tools could assist the preservation of culturally sensitive works of art for future generations, and provide new insights into works of art and the artists who created them.

From vital signs to clinical outcomes for patients with sepsis: a machine learning basis for a clinical decision support system

OBJECTIVE

To develop a decision support system to identify patients at high risk for hyperlactatemia based upon routinely measured vital signs and laboratory studies.

MATERIALS AND METHODS

Electronic health records of 741 adult patients at the University of California Davis Health System who met at least two systemic inflammatory response syndrome criteria were used to associate patients' vital signs, white blood cell count (WBC), with sepsis occurrence and mortality. Generative and discriminative classification (naïve Bayes, support vector machines, Gaussian mixture models, hidden Markov models) were used to integrate heterogeneous patient data and form a predictive tool for the inference of lactate level and mortality risk.

RESULTS

An accuracy of 0.99 and discriminability of 1.00 area under the receiver operating characteristic curve (AUC) for lactate level prediction was obtained when the vital signs and WBC measurements were analysed in a 24 h time bin. An accuracy of 0.73 and discriminability of 0.73 AUC for mortality prediction in patients with sepsis was achieved with only three features: median of lactate levels, mean arterial pressure, and median absolute deviation of the respiratory rate.

DISCUSSION

This study introduces a new scheme for the prediction of lactate levels and mortality risk from patient vital signs and WBC. Accurate prediction of both these variables can drive the appropriate response by clinical staff and thus may have important implications for patient health and treatment outcome.

CONCLUSIONS

Effective predictions of lactate levels and mortality risk can be provided with a few clinical variables when the temporal aspect and variability of patient data are considered.

Fibronectin and vitronectin promote human fetal osteoblast cell attachment and proliferation on nanoporous titanium surfaces

Improvements in osteoconduction of implant biomaterials require focusing on the bone-implant interface, which is a complex multifactorial system. Surface topography of implants plays a crucial role at this interface. Nanostructured surfaces have been shown to promote serum protein adsorption and osteoblast adhesion when compared to micro-structured surfaces for bone-implant materials. We studied the influence of the serum proteins fibronectin and vitronectin on the attachment and proliferation of osteoblasts onto nanostructured titania surfaces. Human fetal osteoblastic cells hFOB 1.19 were used as model osteoblasts and were grown on nanoporous TiO2 templates, using Ti6AI4V and commercially pure Ti substrates as controls. Results show a significant increase in cell proliferation'on nanoporous TiO2 over flat substrates. Initial cell attachment data exhibited a significant effect by either fibronectin or vitronectin on cell adhesion at the surface of any of the tested materials. In addition, the extent of cell adhesion was significantly different between the nanoporous TiO2 and both Ti6AI4V and commercially pure Ti substrates, with the first showing the highest surface coverage. There was no significant difference on osteoblast attachment or proliferation between the presence of fibronectin or vitronectin using any of the material substrates. Taken together, these results suggest that the increase in osteoblast attachment and proliferation shown on the nanoporous TiO2 is due to an increase in the adsorption of fibronectin and vitronectin because of the higher surface area and to an enhanced protein unfolding, which allows access to osteoblast binding motifs within these proteins.

MRI diffusion tensor tracking of a new amygdalo-fusiform and hippocampo-fusiform pathway system in humans

PURPOSE

To use MRI diffusion-tensor tracking (DTT) to test for the presence of unknown neuronal fiber pathways interconnecting the mid-fusiform cortex and anteromedial temporal lobe in humans. Such pathways are hypothesized to exist because these regions coactivate in functional MRI (fMRI) studies of emotion-valued faces and words, suggesting a functional link that could be mediated by neuronal connections.

MATERIALS AND METHODS

A total of 15 normal human subjects were studied using unbiased DTT approaches designed for probing unknown pathways, including whole-brain seeding and large pathway-selection volumes. Several quality-control steps verified the results.

RESULTS

Parallel amygdalo-fusiform and hippocampo-fusiform pathways were found in all subjects. The pathways begin/end at the mid-fusiform gyrus above the lateral occipitotemporal sulcus bilaterally. The superior pathway ends/begins at the superolateral amygdala. The inferior pathway crosses medially and ends/begins at the hippocampal head. The pathways are left-lateralized, with consistently larger cross-sectional area, higher anisotropy, and lower minimum eigenvalue (D-min) on the left, where D-min assesses intrinsic cross-fiber diffusivity independent of curvature.

CONCLUSION

A previously-undescribed pathway system interconnecting the mid-fusiform region with the amygdala/hippocampus has been revealed. This pathway system may be important for recognition, memory consolidation, and emotional modulation of face, object, and lexical information, which may be disrupted in conditions such as Alzheimer's disease.