End to End Optimized Tiny Learning for Repositionable Walls in Maze Topologies
Danilo Pietro Pau, Stefano Colella, Claudio Marchisio · 2023 IEEE International Conference on Consumer Electronics (ICCE) · 2023
In the context of Tiny Machine Learning, the adoption of Deep Reinforcement Learning has been severely prevented due to the computationally expensive demand of such method. This work proposes the application of a Deep Reinforcement Learning algorithm used to train a tiny Convolutional Neural Network such that can be deployed on a cheap, off-the-shelf Microcontroller Unit (MCU) aimed to solve a physical, electrically actuated tilting maze with repositionable walls. The method proposed to train the agent take advantage the lost games to avoid overfitting on certain trajectories. The adopted neural network has been quantized to 8-bits to fit into the MCU embedded memory, comparing the performances against the non-quantized networks and the variable number of repositionable walls. The trained policy networks achieved encouraging results, with win rates between 87% and 99% depending on the difficulty of the task and the model size of the network, with low inference times of about 10 ms. This result, in principle, enables an End-to-End Learning approach able to autonomously sense from the external world, and then compute and execute the actions to solve the physical maze prototype with variable numbers of repositionable walls across games.