The Deep Feed-Forward Gaussian Process: An Effective Generalization to Covariance Priors
Melih Kandemir, Fred A. Hamprecht · Neural Information Processing Systems · 2015
We explore ways of applying a prior on the covariance matrix of a Gaussian Process (GP) in order to increase its expressive power. We show that two well-known covariance priors, Wishart Process and Inverse Wishart Process, boil down to a two-layer feed-forward network of GPs with a particular kernel function on the neuron at the output layer. Both of these models perform supervised manifold learning and target prediction jointly. Also, the resultant kernel functions of both of these priors lead to feature maps of nite dimensionality. Motivated by this fact, we promote replacing these kernels with the Radial Basis Function (RBF), which gives an innite dimensional feature map, enhancing the model exibility. We demonstrate on one benchmark task and two challenging medical image analysis tasks that our GP network with RBF kernel largely outperforms the earlier two covariance priors. We show also that it straightforwardly allows non-linear combination of dierent data views, leading to state-of-the-art multiple kernel learning only as a by-product.