Novel Unsupervised Cluster Reinforcement Q-Learning in Minimizing Energy Consumption of Federated Edge Cloud
Guruh Fajar Shidik, Oki Setiono, Edi Jaya Kusuma, Lekso Budi Handoko, Pulung Nurtantio Andono, Mohd Faizal Abdollah · IEEE Access · 2025
As global demand for cloud computing rises, green computing has become essential. Federated Edge Cloud (FEC) offers improved energy efficiency compared to traditional infrastructures, yet managing distributed energy consumption remains a challenge. This research introduces an Unsupervised Cluster Reinforcement Q-Learning method in FEC (UCRL-FEC), which integrates Fuzzy C-Means (FCM) or K-Means clustering to identify migratable Virtual Machines (VMs) from overloaded hosts. The method enhances energy efficiency and workload balance by incorporating a modified reward function in Q-Learning. Experimental evaluations demonstrate that UCRL-FEC reduces energy consumption (EC) up to 1.07%, contributing to lower operational costs and a reduced carbon footprint, which is critical for large-scale cloud environments. In terms of Service Level Agreement Time per Active Host (SLATAH), UCRL-FEC achieves an improvement up to 1.56% over the baseline method, demonstrating enhanced efficiency in managing active host resources. Additionally, system stability improves with up to 9.68% reduction in Performance Degradation due to Migration (SLA-PDM), effectively minimizing service disruptions and ensuring efficient workload management. Furthermore, the method reduces overall Service Level Agreement Violations (SLAV) up to 6.06%, indicating enhanced service reliability and optimized resource allocation. A Friedman test confirms statistically significant improvements in energy efficiency, workload distribution, and system stability over baseline methods. These advancements prevent resource overutilization, enhance workload management, and extend hardware lifespan, fostering sustainable cloud operations. UCRL-FEC balances energy efficiency, performance, and scalability through dynamic resource optimization, making it a viable solution for intelligent VM management in modern cloud-edge infrastructures.