Adaptive Coding and Prediction of Sources With Large and Infinite Alphabets
Boris Ya. Ryabko, Jaakko T. Astola, Alex Gammerman · IEEE Transactions on Information Theory · 2008
The problem of predicting a sequence x1,x2,. . . generated by a discrete source with unknown statistics is considered. Each letter xt+1is predicted using the information on the word x1x2hellip xtonly. This problem is of great importance for data compression, because of its use to estimate probability distributions for PPM algorithms and other adaptive codes. On the other hand, such prediction is a classical problem which has received much attention. Its history can be traced back to Laplace. We address the problem where the sequence is generated by an independent and identically distributed (i.i.d.) source with some large (or even infinite) alphabet and suggest a class of new methods of prediction.