Learning Entropy as a Learning-Based Information Concept

Introduction to Extreme Seeking Entropy

Recently, the concept of evaluating an unusually large learning effort of an adaptive system to detect novelties in the observed data was introduced. The present paper introduces a new measure of the learning effort of an adaptive system. The proposed method also uses adaptable parameters. Instead of a multi-scale enhanced approach, the generalized Pareto distribution is employed to estimate the probability of unusual updates, as well as for detecting novelties. This measure was successfully tested in various scenarios with (i) synthetic data, (ii) real time series datasets, and multiple adaptive filters and learning algorithms. The results of these experiments are presented.

Entropy Measures for Data Analysis: Theory, Algorithms and Applications

Entropies and entropy-like quantities are playing an increasing role in modern non-linear data analysis and beyond [...]