Stable activation-based regression with localizing property
Jae‐Kyung Shin, Jae‐Hwan Jhong, Ja‐Yong Koo · Communications for Statistical Applications and Methods · 2021
In this paper, we propose an adaptive regression method based on the single-layer neural network structure.We adopt a symmetric activation function as units of the structure.The activation function has a flexibility of its form with a parametrization and has a localizing property that is useful to improve the quality of estimation.In order to provide a spatially adaptive estimator, we regularize coefficients of the activation functions via ℓ 1 -penalization, through which the activation functions to be regarded as unnecessary are removed.In implementation, an efficient coordinate descent algorithm is applied for the proposed estimator.To obtain the stable results of estimation, we present an initialization scheme suited for our structure.Model selection procedure based on the Akaike information criterion is described.The simulation results show that the proposed estimator performs favorably in relation to existing methods and recovers the local structure of the underlying function based on the sample.