On the minimum description length principle for sources with piecewise constant parameters
Neri Merhav · IEEE Transactions on Information Theory · 1993
Universal lossless coding in the presence of finitely many abrupt changes in the statistics of the source, at unknown points, is investigated. The minimum description length (MDL) principle is derived for this setting. In particular, it is shown that, for any uniquely decipherable code, for almost every combination of statistical parameter vectors governing each segment, and for almost every vector of transition instants, the minimum achievable redundancy is composed from 0.5 log n/n bits for each unknown segmental parameter and log n/n bits for each transition, where n is the length of the input string. This redundancy is shown to be attainable by a strongly sequential universal encoder, i.e., an encoder that does not utilize the knowledge of a prescribed value of n.>