Two-Policy Cooperative Transfer for Alleviation of Sim-to-Real Gap
Liangdong Wu, Fangzhou Xiong, Zhiyong Liu · 2022
The main difficulty of sim-to-real is the reality gap between the source domain and the target domain. In order to solve it, various methods where domain randomization is the mainstream have been emerged, whose essence is to make the single policy more robust. In contrast, we propose a novel transfer method, namely two-policy cooperative transfer, whose core is that one policy (task policy) is used to complete the task and another policy (gap policy) aims to assist the former to cover the gap, hence we can focus on the training of task and the overcoming of gap respectively. Based on this method, the setting of the learning objective of gap policy depends on the transfer situation of deploying task policy into real system, besides how to conduct the cooperation of the both lies in the threshold reflecting gap and the coupling of output actions of two policies. For the typical contact-rich gap in the dynamics field, we design an adaptive object pushing experiment based on UR3 robot, and verify the effectiveness of the proposed method.