Relative entropy and quantizer mismatch
Robert M. Gray, Tamás Linder · 2003
Mismatch in vector quantization or source coding is a measure of the performance loss resulting when a code optimized for one source is applied to another. In lossless compression, the relative entropy between the sources for which the code is designed and the source to which it is actually applied measures mismatch. Recent results in high rate vector quantization theory extend these ideas to the relative entropy between continuous distributions as a measure of quantizer mismatch. The results and some of their applications and examples are described, including the use of relative entropy in a distortion measure for Lloyd clustering for the design of Gauss mixture models and classified vector quantizers.