Federated Learning with MLPerfTiny Tasks and Server-side Momentum
Lawrence Roman A. Quizon, Anastacia B. Alvarez · 2024
Federated learning can bring significant benefits to edge IoT systems in their scalability, efficiency, and application space by increasing the amount of computing for the nodes while decreasing the amount of network traffic required. On the other hand, the rise of efficient techniques in TinyML has been a significant boon for ultra low-power IoT ML sensors. Sadly, works on federated learning either use standard networks that are too large for TinyML devices, or are applied to relatively easier tasks to compensate for the performance degradation. In this work, we model federated learning on all four of the MLPerfTiny tasks with their respective baseline models and show that applying federated learning to TinyML models causes significant performance degradation. We also show that the performance degradation is exacerbated when the number of nodes increases. Finally, to address the performance degradation without compromising the original task or increasing the computational load for the client devices, we propose adding momentum to the server-side learning optimizer and show that this significantly mitigates the performance degradation effect, again reaching MLPerfTiny standards on all tasks.