Multi-Finger Grasping with Behavioral Cloning and SAC Fine-Tuning
Suhas Kadalagere Sampath, Bozhen He, Martin J. Pearson, Hao Wu, Cunjia Liu, Chenguang Yang, Ning Wang · 2025
We introduce a learning-based framework for dexterous multi-finger robotic grasping utilising a dexterous hand within the PyBullet simulation environment. Our approach relies exclusively on object and hand pose inputs, deliberately avoiding any visual information during both training and inference. Through a Leap Motion-based teleoperation interface, we gathered over 50 successful expert demonstrations involving both rigid and soft objects, capturing hand-object poses and joint-level finger targets. We trained a lightweight behavioral cloning (BC) model to translate object identity and spatial configuration into finger joint positions. While the BC policy performed effectively on familiar object configurations, it faced challenges in generalising to randomised or unseen object positions. To overcome this limitation, we initialised a Soft Actor-Critic (SAC) policy using the pretrained BC model and fine-tuned it in a physics-based environment with sparse binary rewards. The final SAC policy achieved over 60% success on held-out objects and demonstrated robust generalisation across both rigid and deformable object categories. This pose-driven, vision-free framework provides a scalable foundation for reliable, closed-loop dexterous grasping.