Model Selection: Selecting the Architecture of the Network
Antonis Alexandridis, Achilleas D. Zapranis · 2014
This chapter describes the model selection procedure. One of the crucial steps is to identify the correct topology of the network. The usual approaches proposed to select the architecture of the wavelet network are early stopping, regularization, and pruning. The idea behind minimum prediction risk is to estimate the out-of-sample performance of incrementally growing networks. Information criteria are used in estimation of the number of parameters in linear models. Instead of using information criteria, resampling schemes can also be used to obtain an estimate of the prediction risk. The chapter describes two resampling schemes: bootstrapping and cross-validation. To find an algorithm that works well with wavelet networks and leads to a good estimation of prediction risk, the chapter compares the various criteria as well as the sampling techniques. For some applications where real-time responses of the wavelet network are crucial, online approaches, can be useful.