An achievable bound for optimal noiseless coding of a random variable (Corresp.)
Erik I. Verriest · IEEE Transactions on Information Theory · 1986
For a discreteN-valued random variable (Npossibly denumerably infinite) Leung-Yan-Cheong and Cover have given bounds for the minimal expected length of a one-to-one (not necessarily uniquely decodable) codeL_{1:1}=\sum_{i=1}^{N} p_{i} \log \left( \frac{1}{2} + 1 \right).It is shown that the best possible case occurs for certain denumerably infinite sets of nonzero probabilities. This absolute bound is related to the Shannon entropyHof the distribution by(h (\cdot)is the binary entropy function).