QARMA-FL: Quality-Aware Robust Model Aggregation for Mobile Crowdsourcing
Shehan Edirimannage, Charitha Elvitigala, Ibrahim Khalil, Primal Wijesekera, Xun Yi · IEEE Internet of Things Journal · 2023
Over the past few years, the improved detection and processing features of Internet-of-Things (IoT) devices have opened the doors to several mobile crowdsourcing applications. Federated Learning (FL) is being seen as an attractive framework to address the data privacy concerns of mobile users in the context of crowdsourcing. In FL on a crowdsourcing platform, constructing an effective deep neural network (DNN) is challenging. This is primarily because the quality of the global model depends on the local model quality, which can vary greatly due to differences in the computational resources, data quantity, and data quality provided by each worker. To address these challenges, we propose QARMA-FL: Quality-aware robust model aggregation for federated learning in crowdsourcing applications, where we select the local model for aggregation based on its quality and performance. We also propose a model-quality-aware incentive mechanism to reward workers, based on their contribution to model training. Our model selection and incentive mechanism is capable of detecting Free Rider attacks, identifying workers who benefit from others contributions without contributing themselves. Most existing evaluations of FL in mobile crowdsourcing studies are not based on the real-world FL scenarios. Therefore, we evaluate QARMA-FL alongside a baseline FL model in a quantity-skew, non-IID data setup where different workers contribute varying amounts of data for model training. Our diverse experiments validated QARMA-FLs performance, demonstrating its ability to efficiently aggregate models in mobile crowdsourcing scenarios, reaching baseline results with a reduced worker participation by 40% to 60%.