A 94Hz Inference and 7.4mJ/Epoch Fine-Tune Edge SoC for Diffusion-Based Robot Manipulation with Speculation and Disturbance Enhancement
Shuyuan Zhang, Xizhu Chen, Cheng Peng, Huazhong Yang, Yongpan Liu, Hongyang Jia · 2025
We present an edge SoC for diffusion-transformer-based action generation (DiTAG) in robot manipulation featuring both low-latency inference and high-fidelity on-device finetune. The substantial challenges of their edge acceleration are overcome by a speculative parallel inference and disturbanceenhanced low-bit fine-tune architecture. The 28nm prototype integrating the quad-core accelerator with CPU shows 10.6ms inference,$36.8 \times$better than an edge GPU, with 7.88TOPS/W system energy efficiency, and$7.4 \text{mJ} /$epoch on-device finetune with minimal accuracy loss at normal voltage.