A Multiagent Learning based Hierarchical Distribute Control Framework with Low Communication Costs in Large Scale AUG Networks
Yimo Yan, Shuhe Zhang, Zhihao Feng, Tianlin Wang, Weizheng Sun, Honglin Bao · 2017
In this paper, a multiagent learning based hierarchical distributed framework is proposed to control large scale Autonomous Underwater Glider (AUG) networks. In this framework, multi-AUG system is divided into Management AUG and Work AUG based on their capabilities. Hierarchical distributed architecture signifies that ship station controls Management AUGs and Management AUGs control Work AUG clusters respectively, and information is diffused hierarchically. We propose a novel Q-value based supervising model with pre-learning additionally. AUGs learn collectively from local interactions with their Neighbor AUGs using multiagent Q-learning method. Supervision will occur in hierarchical communication process to diffuse AUGs' learnt information based on their underlying roles. These two approaches decrease communication costs efficiently. Coordination will evolve from the combination of bottom pre-learning and up supervising. Extensive experiments are carried out to test the proposed framework with different topological and parameter settings. We also consider robustness of this framework when a small amount of AUGs has sudden failures. Experimental results show that our framework is indeed effective to distributed control large scale AUG networks and it is quite robust handling sudden failures.