A Universal Learning Rule That Minimizes Well-Formed Cost Functions
Inmaculada Mora-Jiménez, Jesús Cid‐Sueiro · IEEE Transactions on Neural Networks · 2005
In this paper, we analyze stochastic gradient learning rules for posterior probability estimation using networks with a single layer of weights and a general nonlinear activation function. We provide necessary and sufficient conditions on the learning rules and the activation function to obtain probability estimates. Also, we extend the concept of well-formed cost function, proposed by Wittner and Denker, to multiclass problems, and we provide theoretical results showing the advantages of this kind of objective functions.