Flexibility and Privacy: A Multi-Head Federated Continual Learning Framework for Dynamic Edge Environments
Chunlu Chen, Kevin I‐Kai Wang, Peng Li, Kouichi Sakurai · 2023
Federated Learning (FL) and the Internet of Things (IoT) have revolutionized data processing and analysis, overcoming the traditional limitations of cloud computing. However, traditional machine learning strategies lead to increased costs and catastrophic forgetting due to model retraining with new datasets. Continual learning has been proposed to counter this, enabling models to adapt to new data while preserving previous knowledge, which is beneficial for dynamic edge environments. Despite these advantages, the retention of previous knowledge during the continual learning process may lead to the information leakage. To address the inherent challenges of multitask scenarios, we present a Federated Continual Learning (FCL) framework that integrates the privacy-preserving benefits of Federated Learning (FL) into a continual learning system, ensuring both continual learning and privacy preservation in edge computing data processing and analysis. Specifically, our architecture introduces dedicated fully-connected layers for each task. This architecture ensures that distinctive features pertinent to each task are not only captured but also preserved throughout the model’s lifespan. Within our framework, data is processed via task-specific layers. Subsequently, the final label is determined by associating it with the paramount prediction value, thus capitalizing on the model’s comprehensive knowledge reservoir to bolster prediction accuracy. We subjected our FCL framework to rigorous validation using two benchmark datasets: MNIST and CFAR-10. Experimental outcomes unequivocally substantiate the efficacy of our proposed methodology.