Federated Learning Using GPT-4 Boosted Particle Swarm Optimization for Compact Neural Architecture Search
Di Wang · Journal of Advances in Information Technology · 2024
In response to the growing need for privacypreserving mobile intelligence, this study introduces a new approach that combines Generative Pre-trained Transformer 4 (GPT-4), a state-of-the-art large language model, with Particle Swarm Optimization (PSO) in a twostep process.This method is designed to find efficient neural network structures in federated learning and address issues like high communication costs and unstable network conditions.Leveraging the prowess of GPT-4 for initial population guidance in the Neural Architecture Search (NAS) process, our approach focuses on optimizing neural network architectures that demand minimal data exchange between clients and servers.This is achieved through a variable-length PSO encoding and decoding mechanism at the upper level, ensuring not only a thorough search for efficient architectures but also their optimization for compactness and effectiveness.Additionally, a standard PSO technique is applied at the lower level to optimize neural network weights, thus boosting model performance with reduced communication load.Our methodology's superiority is demonstrated via benchmark comparisons with FedAvg and FedPSO on the CIFAR-10 dataset, under both normal and compromised network scenarios.