How to Pretend That Correlated Variables Are Independent by Using Difference Observations
Christopher K. I. Williams · Neural Computation · 2004
In many areas of data modeling, observations at different locations (e.g.,time frames or pixel locations) are augmented by differences of nea r by observations (e.g., delta features in speech recognition, Gabor jets in image analysis). These augmented observations are then often modeled as being independent. How can this make sense?We provide two interpretations,showing (1) that the likelihood of data generated from an auto regressive process can be computed in terms of "independent" augmented observations and (2) that the augmented observations can be given a coherent treatment in terms of the products of experts model (Hinton, 1999).