MDL transduction
Liwei Wang, Jufu Feng · International Conference on Machine Learning and Cybernetics · 2005
Transduction takes a set of training samples and aims at estimating class labels of given examples in one step as opposed to the traditional induction, which involves an intermediate learning step. The background philosophy of transduction is that one should not reduce an easier task (estimating labels of given examples) to a substantially more complex problem (learning a model). This paper proposes new scheme for transductive inference, which we call MDL transduction. It labels the given examples so that the stochastic complexity of the whole data is minimized. In the sense of minimum description length, MDL transduction outperforms induction in both generative and discriminative methods. A key property of MDL transduction is that it learns nothing about the model. This highly agrees with the afore-mentioned philosophy. Relation to transductive SVM (TSVM) is also discussed. We show that TSVM is an approximation of MDL transduction with discriminant models.