Consensus-Driven Hyperparameter Optimization for Accelerated Model Convergence in Decentralized Federated Learning
Anam Nawaz Khan, Qazi Waqas Khan, Atif Rizwan, Rashid Ahmad, Do‐Hyeun Kim · Internet of Things · 2024
Decentralized Federated Learning (DFL) enables collaborative model training across multiple devices without relying on a central server, preserving data privacy and achieving full decentralization. However, optimizing hyperparameters (HPs) in DFL presents significant challenges due to system and statistical heterogeneity and the lack of centralized coordination. Existing Hyperparameter Optimization (HPO) methods for Federated Learning (FL) typically rely on a central server, making them unsuitable for fully decentralized environments. These methods face scalability issues, high communication and computation overhead, and limited adaptability to node-specific requirements. The need for serverless, fully decentralized HPO is particularly critical in scenarios demanding minimal communication and efficient resource usage. To address these challenges, we propose the single-pass Decentralized Federated Hyperparameter Optimization framework (DFed-HPO), which integrates three advanced HPO strategies for computation and communication-efficient HP optimization. DFed-HPO includes three hyperparameter aggregation mechanisms: MetaRegress Aggregator (MA), Consensus Aggregator (CA), and Fusion Aggregator (FA). MA uses meta-learning principles to predict the performance of new configurations with minimal communication, while CA and FA enhance optimization and aggregation based on model similarity and consensus-building among nodes. By conducting HPO in a single pass, DFed-HPO reduces communication overhead while achieving robust hyperparameter consensus. We validated DFed-HPO on MNIST and the Electricity dataset, demonstrating its potential in energy management applications, including green hydrogen production using renewable energy systems . Results show that DFed-HPO improves model performance, accelerates convergence, and adapts to non-IID data, offering a scalable and efficient solution for decentralized AI networks in resource-constrained settings.