A piecewise linear recurrent neural network structure and its dynamics
Xiao Liu, Tülay Adalı, Levent Demirekler · 2002
We present a piecewise linear recurrent neural network (PL-RNN) structure by combining the canonical piecewise linear function with the autoregressive moving average (ARMA) model such that an augmented input space is partitioned into regions where an ARMA model is used in each. The piecewise linear structure allows for easy implementation, and in training, allows for use of standard linear adaptive filtering techniques based on gradient optimization and description of convergence regions for the step-size. We study the dynamics of PL-RNN and show that it defines a contractive mapping and is bounded input bounded output stable. We introduce application of PL-RNN to channel equalization and show that it closely approximates the performance of the traditional RNN that uses sigmoidal activation functions.