Complex valued recurrent neural networks for noncircular complex signals
Danilo P. Mandic · 2009
This paper uses new developments in the statistics of complex variable and recent results on the duality between the bivariate and complex calculus to provide a unified design of complex valued temporal neural networks. For generality, the case of recurrent neural networks is addressed in detail, as they simplify into feedforward networks upon cancellation of the feedback. The use of CopfRopf calculus provides a convenient framework for the calculation of gradients of real functions of complex variables (cost functions) which do not obey the Cauchy-Riemann conditions. Further, the analysis is based on so called augmented complex statistics, to provide a rigorous treatment of complex noncircularity and nonlinearity, thus avoiding the deficiencies inherent in several mathematical shortcuts typically used in the treatment of complex random signals. The complex models addressed in this work, are based on widely linear nonlinear autoregressive moving average (NARMA) models and are shown to be suitable for processing the generality of complex signals, both second order circular (proper) and noncircular (improper).