Improving Model Accuracy in Federated Learning with Mish Activation Function and the FedAvg-RAdam Optimization Algorithm
Samane Dandani, Mohammad Hossein Yaghmaee · 2024
Federated Learning (FL) has emerged as a promising paradigm for cooperatively training Deep Neural Networks (DNNs) on mobile devices while ensuring client privacy for decentralized data. FL aims to train models across multiple computational units (users), enabling clients to communicate with a central shared server without sharing their data samples. This approach leverages the computational power of all clients while preserving their privacy to achieve an accurate global model. However, prior studies have revealed that non-independent and identically distributed (non-IID) client data can impair the convergence speed of FL algorithms. Moreover, while most research FL concentrates on assessing the accuracy of the global model, improving the accuracy of individual user models (UA) is often crucial, especially in applications like personalized content recommendation. To address these challenges, this study introduces the FedAvg-RAdam optimization algorithm and the Mish activation function into federated learning, particularly within the context of Multi-Task Federated Learning (MTFL), aiming to expedite convergence and enhance accuracy. Experimental results conducted on the MNIST dataset demonstrate that this approach significantly enhances accuracy compared to the FedAvg-Adam optimization strategy across various scenarios.