On Optimal Data Compression in Multiterminal Statistical Inference
Шун-ичи Амари · IEEE Transactions on Information Theory · 2011
The multiterminal theory of statistical inference deals with the problem of estimating or testing the correlation of letters generated from two (or many) correlated information sources under the restriction of a certain transmission rate for each source. A typical example is two binary sources with joint probabilityp(x,y) where the correlation ofxandyis to be tested or estimated. Givenniid observationsxn=x1...xnandyn=y1...yn, onlyk=rn(0rkletters ofxnandyn. A simpler problem is the helper case where the optimal data compression ofxnis searched for under the condition that all ofynare transmitted. It is a long standing problem to determine if there is a better data compression scheme than this simple scheme of sending firstkletters. The present paper searches for the optimal data compression under the framework of linear-threshold encoding and shows that there is a better data compression scheme depending on the value of correlation. To this end, we evaluate the Fisher information in the class of linear-threshold compression schemes. It is also proved that the simple scheme is optimal whenxandyare independent or their correlation is not too large.