Can finite samples detect singularities of real-valued functions?
Shai Ben-David · 1992
Consider the following type of problem: There is an unknown function, f : R n ! R m , there is also a black-box that on query x (2 R n ) returns f(x). Is there an algorithm that, using probes to the black-box, can figure out analytic information about f? (For an example: "Is f a polynomial? ", "Is f a second order differentiable at x = (0; 0; : : : ; 0)?" etc.). Clearly, for examples as these, if we bound the number of probes an algorithm has to settle for, no algorithm can carry the task. On the other hand, if one allows an infinite iteration of a `probe compute and guess' process, then, (quite surprisingly) for many such questions, there are algorithms that are guaranteed to be correct in all but finitely many of their guesses. We call such questions Decidable In the Limit, (DIL). We analyze the class of DIL problems and provide a necessary and sufficient condition for the membership of a decision problem in this class. We offer an algorithm for any DIL problem, and apply it to several types of learning tasks. We introduce a an extension of the usual Inductive Inference learning model - Inductive Inference with a Cheating Teacher. In this model the teacher may choose to present to the learner, not only a language belonging to the agreed - upon family of languages, but also an arbitrary language outside this family. In such a case we require that the learner will be able to eventually detect the faulty choice made by the teacher. We show that such strong type of learning is possible, and there exist learning algorithms that will fail only on arbitrarily small sets of faulty languages. Furthermore, if an a-priori probability distribution P , according to which f is being chosen, is available to the algorithm, then it can be strengthened into a finite A prelimi...