Comment on "Rényi entropy yields artificial biases not in the data and incorrect updating due to the finite-size data"

Calibration Invariance of the MaxEnt Distribution in the Maximum Entropy Principle

The maximum entropy principle consists of two steps: The first step is to find the distribution which maximizes entropy under given constraints. The second step is to calculate the corresponding thermodynamic quantities. The second part is determined by Lagrange multipliers’ relation to the measurable physical quantities as temperature or Helmholtz free energy/free entropy. We show that for a given MaxEnt distribution, the whole class of entropies and constraints leads to the same distribution but generally different thermodynamics. Two simple classes of transformations that preserve the MaxEnt distributions are studied: The first case is a transform of the entropy to an arbitrary increasing function of that entropy. The second case is the transform of the energetic constraint to a combination of the normalization and energetic constraints. We derive group transformations of the Lagrange multipliers corresponding to these transformations and determine their connections to thermodynamic quantities. For each case, we provide a simple example of this transformation.

When Shannon and Khinchin meet Shore and Johnson: Equivalence of information theory and statistical inference axiomatics

We propose a unified framework for both Shannon-Khinchin and Shore-Johnson axiomatic systems. We do it by rephrasing Shannon-Khinchine axioms in terms of generalized arithmetics of Kolmogorov and Nagumo. We prove that the two axiomatic schemes yield identical classes of entropic functionals-the Uffink class of entropies. This allows to re-establish the entropic parallelism between information theory and statistical inference that has seemed to be "broken" by the use of non-Shannonian entropies.