Real-Time Performance Benchmarking of TinyML Models in Embedded Systems (PICO: Performance of Inference, CPU, and Operations)
Abhishek Dey, Saurabh Srivastava, Gaurav Kumar Singh, Robert G. Pettit · 2025
This paper presents PICO-TINYML-benchmark, a modular and platform-agnostic framework for benchmarking the real-time performance of TinyML models on resourceconstrained embedded systems. Evaluating key metrics such as inference latency, CPU utilization, memory efficiency, and prediction stability, the framework provides insights into computational trade-offs and platform-specific optimizations. We benchmark three representative TinyML models-Gesture Classification, Keyword Spotting, and MobileNet V2-on two widely adopted platforms, BeagleBone AI64 and Raspberry Pi 4, using realworld datasets. Results reveal critical trade-offs: the BeagleBone AI64 demonstrates consistent inference latency for AI-specific tasks, while the Raspberry Pi 4 excels in resource efficiency and cost-effectiveness. These findings offer actionable guidance for optimizing TinyML deployments, bridging the gap between theoretical advancements and practical applications in embedded systems.