LaVFL: Efficient Verifiable Federated Learning for Large Language Models
Tianyou Zhang, Haiyang Yu, Zhen Yang, Yuwen Chen, Shui Yu · IEEE Transactions on Dependable and Secure Computing · 2025
Federated Learning (FL) represents a distributed machine learning approach, enabling the joint training of a global model through the aggregation of gradients from participating clients without necessitating the exchange of raw data. Prior research has explored methods for verifying the correctness of aggregation in this context and mitigating the overhead associated with the verification process. Nonetheless, the advent of Large Language Models (LLMs), with their parameters numbering in the billions, presents ongoing challenges in devising efficient verification mechanisms in FL for large models. In this paper, we propose an innovative Efficient Verifiable Federated Learning scheme${\sf LaVFL}$, which focusing on addressing the verification challenges incurred by LLM. Specifically, we propose an efficient layer-by-layer verification approach for LLMs by designing a Convolution Gradient Compression (CGC) method without compromising model accuracy. Additionally, to minimize computational and communication overheads, we propose an efficient verification strategy PGS, namely, a Probabilistic Gradient Sampling strategy, which aims to reduce the gradient dimensions for each round of verification while ensuring a high probability of comprehensive verification. We implement a prototype of${\sf LaVFL}$, and extensive experimental results demonstrate that${\sf LaVFL}$achieves over a$300 \times$speedup in the aggregation verification phase and reduces communication overheads by more than 75%, compared to VeriFL under the same experimental setup.