On-line AdaTron learning of unlearnable rules
Jun-ichi Inoue, Hidetoshi Nishimori · Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics · 1997
We study the on-line AdaTron learning of linearly nonseparable rules by a simple perceptron. Training examples are provided by a perceptron with a nonmonotonic transfer function that reduces to the usual monotonic relation in a certain limit. We find that, although the on-line AdaTron learning is a powerful algorithm for the learnable rule, it does not give the best possible generalization error for unlearnable problems. Optimization of the learning rate is shown to greatly improve the performance of the AdaTron algorithm, leading to the best possible generalization error for a wide range of the parameter that controls the shape of the transfer function.