Robust noiseless source coding through a game theoretical approach
Demetrios Kazakos · IEEE Transactions on Information Theory · 1983
Noiseless coding of a discrete source with partially known statistics is formulated as a two-person game. The payoff is the average codeword length, using Shannon codes. The code designer picks a source probability distribution for the design of the code, while an opponent picks the actual source probability distribution. It is shown that if the class of probability distributions allowed is convex, then there is a saddle point solution which is determined by the maximum entropy distribution of the convex class. The maximum entropy element is derived for three families of source probability mass functions (pmf): a) the class of c-contaminated pmf's; b) the class of pmf's for which each probability is known only through an upper and lower bound; c) the class of pmf's which is a convex hull of a finite number of known pmf's. An extension of the robust noiseless source coding problem for families of sources modeled as first-order Markov chains is discussed.