Information Theory and Language

Noise resistance in communication: Quantifying uniformity and optimality

Over the past few decades, several lines of research have investigated how information is distributed in language. In particular, research has suggested that speakers distribute the information in linguistic utterances as evenly as possible, in order to make the utterance more robust against noise for the hearer. Several studies have provided evidence for this hypothesis in limited linguistic domains, showing that speakers manipulate acoustic and syntactic features by using redundancy to avoid drastic spikes or troughs in information content. However, this previous work doesn't consider information density across entire utterances, and only rarely has the proposed function of noise resistance been directly explored in detail. Here, we introduce a new descriptive statistic (DORM) that quantifies the uniformity of information across an entire utterance. We present this alongside an algorithm (UIDO) that allows us to quantify the contribution made to an utterance's uniformity by the order of its elements, independent of its level of redundancy. Using a simple simulation with data from an English corpus, we show that utterances whose order results in more uniform distributions of information are, in fact, more robust against noise.