On asymptotically optimal methods of prediction and adaptive coding
Boris Ya. Ryabko, Flemming Topsøe · 2002
The problem of predicting a sequence x/sub 1/,x/sub 2/, generated by a discrete source with unknown statistics is considered. Each letter x/sub t+1/ is predicted using information on the word x/sub 1/x/sub 2//spl middot//spl middot//spl middot/x/sub t/ only. To estimate the efficiency of a method of prediction, three quantities are considered: the precision as given by the Kullback-Leibler divergence, the memory size of the program needed to implement the method on a computer and the average time required, measured by the number of binary operations for the prediction of a single letter. A method is presented for which the memory size and the average time is close to the minimum. The results can readily be translated to adaptive coding.