Character n-gram application for automatic new topic identification

An ensemble scheme based on language function analysis and feature engineering for text genre classification

Text genre classification is the process of identifying functional characteristics of text documents. The immense quantity of text documents available on the web can be properly filtered, organised and retrieved with the use of text genre classification, which may have potential use on several other tasks of natural language processing and information retrieval. Genre may refer to several aspects of text documents, such as function and purpose. The language function analysis (LFA) concentrates on single aspect of genres and it aims to classify text documents into three abstract classes, such as expressive, appellative and informative. Text genre classification is typically performed by supervised machine learning algorithms. The extraction of an efficient feature set to represent text documents is an essential task for building a robust classification scheme with high predictive performance. In addition, ensemble learning, which combines the outputs of individual classifiers to obtain a robust classification scheme, is a promising research field in machine learning research. In this regard, this article presents an extensive comparative analysis of different feature engineering schemes (such as features used in authorship attribution, linguistic features, character n-grams, part of speech n-grams and the frequency of the most discriminative words) and five different base learners (Naïve Bayes, support vector machines, logistic regression, k-nearest neighbour and Random Forest) in conjunction with ensemble learning methods (such as Boosting, Bagging and Random Subspace). Based on the empirical analysis, an ensemble classification scheme is presented, which integrates Random Subspace ensemble of Random Forest with four types of features (features used in authorship attribution, character n-grams, part of speech n-grams and the frequency of the most discriminative words). For LFA corpus, the highest average predictive performance obtained by the proposed scheme is 94.43%.

A search index-enhanced feature model for news recommendation

General news recommendations are important but have received limited attention because of the difficulties of measuring public interest. In public search engines, the objects of search terms reflect the issues that interest or concern search engine users. Because of the popularity of search engines, search indexes have become a new measure for describing public interest trends. With the help of a public search index provided by search engines, we construct a news topic search feature and a news object search feature. These features measure the public attention on key elements of the news. In the experiment, we compare various feature models with machine learning algorithms with respect to financial news recommendations. The results demonstrate that the topic search features perform best compared with other feature models. This research contributes to both the feature generation and news recommendation domains.

Updating controlled vocabularies by analysing query logs

Purpose

– Controlled vocabularies play an important role in information retrieval. Numerous studies have shown that conceptual searches based on vocabularies are more effective than keyword searches, at least in certain contexts. Consequently, new ways must be found to improve controlled vocabularies. The purpose of this paper is to present a semi-automatic model for updating controlled vocabularies through the use of a text corpus and the analysis of query logs.

Design/methodology/approach

– An experimental development is presented in which, first, the suitability of a controlled vocabulary to a text corpus is examined. The keywords entered by users to access the text corpus are then compared with the descriptors used to index it. Finally, both the query logs and text corpus are processed to obtain a set of candidate terms to update the controlled vocabulary.

Findings

– This paper describes a model applicable both in the context of the text corpus of an online academic journal and to repositories and intranets. The model is able to: first, identify the queries that led users from a search engine to a relevant document; and second, process these queries to identify candidate terms for inclusion in a controlled vocabulary.

Research limitations/implications

– Ideally, the model should be used in controlled web environments, such as repositories, intranets or academic journals.

Social implications

– The proposed model directly improves the indexing process by facilitating the maintenance and updating of controlled vocabularies. It so doing, it helps to optimise access to information.

Originality/value

– The proposed model takes into account the perspective of users by mining queries in order to propose candidate terms for inclusion in a controlled vocabulary.