A general probabilistic formulation for neural classifiers
T. Adalz · 2002
We use partial likelihood (PL) theory to introduce a general probabilistic framework for the design and analysis of neural classifiers. The formulation allows for the training samples used in the design to have correlations in time, and for use of a wide range of neural network probability models including recurrent structures. We use PL theory to establish a fundamental information-theoretic connection, show the equivalence of likelihood maximization and relative entropy minimization, without making the common assumptions of independent training samples and true distribution information. Large sample optimality properties of PL can also be established under mild regularity conditions which allows adaptive-structure and robust classifier designs by using modified likelihood functions and information-theoretic criteria.