Information about Hyperparameters in Hierarchical Models
Prem K. Goel, Morris H. DeGroot · Journal of the American Statistical Association · 1981
We consider situations in which the prior distribution of a parameter vector θ1 in the distribution of an observable random vector X contains a hyperparameter vector θ2. The experimenter specifies another distribution for θ2 that contains hyperparameters θ3, and so forth. One wants to learn about the hyperparameters at each level of this hierarchical model. We show that for many measures of information, the gain in information decreases as one moves to higher levels of hyperparameters. These results are illustrated for univariate normal models and a general linear hierarchical model. Examples of measures of information are given for which this property does not hold.