Correlation learning based multi-task model and its application
Wei Xu, Jianping LUO, Xia Li, Wenming Cao · JOURNAL OF SHENZHEN UNIVERSITY SCIENCE AND ENGINEERING · 2023
Multi-task learning is a means of learning by combining multiple tasks simultaneously to enhance the model representation and generalization ability. The correlation between tasks is the key factor for the construction of multi-task learning model. In order to solve the problem of inherent conflicts of task differences that can damage the prediction of some tasks, a multi-task learning model based on correlation learning layer (CLL) is proposed. Meanwhile, the proposed multi-task learning model is applied as a new agent model to the Bayesian optimization algorithm to solve expensive optimization problems. A CLL is added behind the traditional multi-task learning network, so that the tasks that have completed the preliminary shared learning can be optimized and carry out the advanced sharing in this layer, so that the knowledge learned from multiple tasks can fully interact with each other. According to different parameter-based sharing mechanisms, the LeNet and radial basis function (RBF) multi-task learning models with correlation layers are constructed. The experiments are conducted on the multi-task version of the Mixed National Institute of Standards and Technology (MNIST) database and the comprehensive data set with controllable task correlation. The experimental results verify the effectiveness of the proposed multi-task learning model based on the correlation layer. Meanwhile, the proposed multi-task learning network as a proxy model is applied to the Bayesian optimization algorithm, which not only reduces the evaluation times of model to target problem, but also enlarges the number of training data exponentially and further improves the model accuracy.