Data-Dependent Regularization
Corduneanu Adrian, Jaakkola Tommi · The MIT Press eBooks · 2006
This chapter considers two ways of representing the topology over examples, either based on complete knowledge of the marginal density or by grouping together examples whose labels should be related. The learning algorithms and sample complexity issues that result from each representation is discussed here. Information regularization is a principle for assigning labels to unlabeled data points in a semi-supervised setting. The broader principle is based on finding labels that minimize the information induced between examples and labels relative to a topology over the examples; any label variation within a small local region of examples ties together the identities of examples and their labels. Such variation should be minimized unless supported directly or indirectly by the available labeled examples.