On the average codeword length of optimal binary codes for extended sources (Corresp.)

Abstract

Although optimal binary source coding using symbol blocks of increasing length must eventually yield a code having an average codeword length arbitrarily close to the source entropy, it is known that the sequence of average codeword lengths need not be nonincreasing. The sequence is, however, bounded above by the average codeword length of the source, and certain subsequences must be nondecreasing. Sufficient conditions are obtained describing sources for which a decrease in average codeword length is achieved when coding pairs of symbols. A sufficient condition specifying sources for which no such decrease is possible is also obtained.