Some Stochastic Properties of Memoryless Individual Sequences
Andrew B. Nobel · IEEE Transactions on Information Theory · 2004
An individual sequence of real numbers is memoryless if no continuous Markov prediction scheme of finite order can outperform the best constant predictor under the squared loss. It is established that memoryless sequences satisfy an elementary law of large numbers, and sliding-block versions of Hoeffding's inequality and the central limit theorem. It is further established that memoryless binary sequences have convergent sample averages of every order, and that their limiting distributions are Bernoulli. Several examples and sources of memoryless sequences are given, and it is shown how memoryless binary sequences may be constructed from aggregating methods for sequential prediction.