Intelligent redundant manipulation for long-horizon operations with multiple goal-conditioned hierarchical learning
Haoran Zhou, Xiankun Lin · Advanced Robotics · 2025
Collaborative robotic arms face significant challenges in achieving high success rates over long-term tasks due to constraints on their degrees of freedom and the inherent complexity of planning. To tackle this challenge, we propose a hierarchical Reinforcement Learning framework, called Multiple Goal-conditioned Hierarchical Learning (MGHL), aimed at addressing the long-horizon task challenges in redundant manipulators. In the framework, we regard a long-horizon operation task as a sequential composition of multiple subtasks. By integrating Multi-Task Learning, MGHL learns these decomposed subtasks in terms of a monolithic policy network. To avoid the difficulties in manual designing a reward function for each subtask, MGHL employs sparse rewards to learn subtasks. However, this implementation will reduce the exploration capability of the policy. A novel exploration strategy with setting a sub-goal for each subtask is proposed to improve the capability in exploration of agents. In experiments, three types of long-horizon manipulation tasks are introduced for evaluation. The experimental results show that the proposed framework can learn multiple subtasks with high success rates and can sequentially achieve the goal of each subtask. It is also indicated from comparisons that MGHL outperforms the comparative state-of-the-art algorithms in solving long-horizon tasks.