Research on Activation Functions in Machine Learning Based Network Coding
Xin Zhang, Yanbo Yang, Baoshan Li, Minchao Li, Teng Li, Jiawei Zhang · 2023
Network coding allows network Intermediate nodes to encode on top of data forwarding, which can improve network transmission efficiency, robustness and security. However, traditional nonlinear network coding suffers from the problems of coding and decoding complexity and implementation difficulties, while the real network environment is often nonlinear. Machine learning-based network coding utilizes neural networks and activation functions for coding and decoding, which solves the complexity and implementation difficulties of traditional nonlinear network coding, where activation functions, as the main factor for introducing nonlinearity, are crucial for machine learning-based network coding. In this paper, the basic ReLU and Sigmoid-type activation functions are chosen to investigate their role by transmitting information over a butterfly network. Firstly, it is pointed out that the presence of activation functions in the absence of noise interference at intermediate nodes can cause compression or loss of coded information, Secondly, the theorem of the role of noise on the activation function is derived in the case of noisy interference at intermediate nodes. Based on this theorem, it is concluded that the ReLU activation function has a better transmission effect when the interference is small, and the Sigmoid activation function has a better transmission effect when the interference is large; meanwhile, it is pointed out that the anti-noise performance of the Sigmoid activation function is better than that of the ReLU activation function.