A Developmental Method that Computes Optimal Networks without Post-Selections
Juyang Weng · 2021
This work is the theory of Post-Selection practices that have been rarely studied. Post-Selections mean selections of systems after the systems have been trained. Post-Selections Using Validation Sets (PSUVS) are wasteful and Post-Selections Using Test Sets (PSUTS) are wasteful and unethical. Both result in systems whose generalization powers are weak. The PSUTS fall into two kinds, machine PSUTS and human PSUTS. The connectionist AI school received criticisms for its “scruffiness” due to a huge number of network parameters and now the machine PSUTS; but the seemingly “clean” symbolic AI school seems more brittle because of its human PSUTS. This paper analyzes why, in deep learning, error-backprop methods with random initial weights suffer from severe local minima, why PSUTS violates well-established research ethics, and publications that used PSUTS should have transparently reported PSUTS. This paper proposes a Developmental Methodology that trains only a single but optimal network for each application lifetime using a new standard for performance evaluation in machine learning, called developmental errors for all networks trained in a project that the selection of the luckiest network depends on, along with Three Learning Conditions: (1) framework restrictions, (2) training experience and (3) computational resources. This paper also discusses how the brain-inspired Developmental Networks (DNs) avoid PSUTS by reporting developmental errors and its maximum likelihood (ML) optimality under the Three Learning Conditions. DNs are not “scruffy” because they are ML-estimators of the observed Emergent Turing Machines at each time during their “lives”. This implies best performance given a limited amount of overall available computational resources for a project.