MIMO Modeling by Learning Explicitly the Projection Space: The Maximum Correlation Ratio Cost Function
Bo Hu, José Carlos Príncipe · IEEE Transactions on Signal Processing · 2021
Maximal correlation measures the statistical dependency between two random variables and has been broadly used by statisticians. In this paper, we present a novel regression framework based on the rich theory of maximal correlation to train a Multiple-input Multiple-output (MIMO) model called the Bank of Wiener models (BWM). Nonlinear time series modeling under maximal correlation unifies the cost and the mapping function under the same mathematical optimization framework. This provides direct optimality (maximal statistical dependence) between BWM model outputs and the desired response, without the restriction of constructing an error signal as conventionally done in regression. Based on the idea of maximal correlation, we propose the Maximal Correlation Algorithm (MCA), which approximates directly the maximal correlation ratio between BWM outputs and the desired response. As a result, MCA modularizes the training of models with hidden layers and avoids the end-to-end training of backpropagation (BP), while improving the equivalent mapping capability. We prove that MCA is a pseudo-concave function, which yields better-behaved parameter optimization. We demonstrate experimentally that MCA is very competitive when compared to a single-hidden-layer MLP trained with BP and the Mean Squared Error (MSE). We also show that the system identification capabilities of MCA are superior to MLPs trained with BP. Hence, MCA has great promise in both nonlinear system identification and machine learning.