Sig-R2ResNet: Residual Network with Signal Processing-refined Residual Mapping, Auto-tuned L1-Regularization with Modified Adam Optimizer for Time Series Classification
Arijit Ukil, Soma Bandyopadhyay, Arpan Pal · 2020
Time Series Classification (TSC) is becoming a challenging and important problem to solve specifically due to the advent of sensor-based applications and Internet of Things (IoT). Residual mapping displays easier and evidently near-optimal learning. In this paper, we extend this notion by incorporating fine-grained refining of the residual learning through augmentation of feature space using gamut of signal processing transformations. Our proposed Sig-R2ResNet refines the learning process by introducing newer representation through signal processing primitives without distorting the residual mapping channel, along with an auto-tuning L1regularization. We further adapt the learning rate decay through learning over the trend of validation loss and modify the network parameter update process of Adam optimizer. The proposed method- Sig-R2ResNet can be viewed as an informal game where training experience is augmented through unsupervised signal processing features while the model growth is controlled by a regularization process and smoother learning convergence is achieved by validation loss dependent learning rate. One of the novelties is that the training signal dynamics control the enhancement of representation complexity, regularization and learning rate adaptation. We perform extensive experiments with diverse datasets from publicly available UCR time series database and demonstrate empirical evidences that our method consistently outperforms the existing benchmark results (creating 59.10% new benchmark results) as well as shows significantly better classification outcome than the current baselines and state-of-the-art algorithms like ResNet, BOSS, COTE.