Optimum estimation via partition functions and information measures
Neri Merhav · 2010
In continuation to a recent work on the statistical-mechanical analysis of optimum estimation in Gaussian noise via its relation to the mutual information (I-MMSE relation), here we propose a more direct relation between optimum estimation and some information measures, which can be viewed as partition functions and hence are amenable to statistical-mechanical analysis. This approach has several advantages, most notably, its applicability to general sources/channels, as opposed to the I-MMSE relation and its variants which hold only for certain classes of channels. We also demonstrate the derivation of the optimum estimator and the MMSE in a few examples. One of them is generalizable to a fairly wide class of sources and channels. For this class, our approach yields an approximate conditional mean estimator and an MMSE formula that has the flavor of a single-letter expression.