Second-order noiseless source coding theorems
Ioannis Kontoyiannis · IEEE Transactions on Information Theory · 1997
Shannon's celebrated source coding theorem can be viewed as a "one-sided law of large numbers". We formulate second-order noiseless source coding theorems for the deviation of the codeword lengths from the entropy. For a class of sources that includes Markov chains we prove a "one-sided central limit theorem" and a law of the iterated logarithm.