Converge Faster, Talk Less: Hessian-Informed Federated Zeroth-Order Optimization
Zhe Li, Bicheng Ying, Zidong Liu, Dong, Chaosheng, Haibo Yang · PubMed · 2025
-bound. Empirically, across diverse LLM fine-tuning benchmarks, HiSo delivers a 1~5× speedup in communication rounds over existing state-of-the-art ZO-FL baselines. This superior convergence not only cuts communication costs but also provides strong empirical evidence that Hessian information acts as an effective accelerator in federated ZO optimization settings.