Gradient of Mutual Information in Linear Vector
Daniel P. Palomar, Sergio Verdú · 2006
This paper considers a general linear vector Gaussian channel with arbitrary signaling and pursues two closely related goals: i) closed-form expressions for the gradient of the mutual in- formation with respect to arbitrary parameters of the system, and ii) fundamental connections between information theory and esti- mation theory. Generalizing the fundamental relationship recently unveiled by Guo, Shamai, and Verdu, we show that the gradient of the mutual information with respect to the channel matrix is equal to the product of the channel matrix and the error covariance ma- trix of the best estimate of the input given the output. Gradients and derivatives with respect to other parameters are then found via the differentiation chain rule. Index Terms—De Bruijn's identity, divergence, Gaussian noise, minimum mean-square error (MMSE), multiple-input multiple- output (MIMO) channels, mutual information, nonlinear estima- tion, precoder optimization.