Elements of Information Theory

Abstract

Information theory deals with the quantitative aspects of storage and transmission of messages. Some of these aspects involve the inner structure of messages: the grammar of their language, the acoustic spectrum of their sound etc. We will not enter into questions of this kind. For us here, messages are elements of a set, and when we handle them, our sole concern is to keep distinct messages distinct, i.e. to make the mappings resulting from our manipulations injective (= one-to-one). Aiming at this, we face two obstacles: of storage or transmission procedures, i.e. distortion of messages during transmission.