Multi-agent Multi-game Entity Transformer: Towards Generalist Models in MARL
Rundong Wang, Weixuan Wang, Xianhan Zeng, Liang Wang, Zhengjie Liang, Yiming Gao, Feiyu Liu, Siqin Li, Xianliang Wang, Fu Qiang, Wei Yang, Lanxiao Huang, L. Zheng, Zinovi Rabinovich, Bo An · 2024
Building large-scale generalist pre-trained models for many tasks is becoming an emerging and potential direction in reinforcement learning (RL).Research such as Gato and Multi-Game Decision Transformer have displayed outstanding performance and generalization capabilities on many games and domains.However, there exists a research blank about developing highly capable and generalist models in multi-agent RL (MARL), which can substantially accelerate progress toward general AI.To fill this gap, we propose Multi-Agent multi-Game ENtity TrAnsformer (MA-GENTA) from the entity perspective as orthogonal research to previous time-sequential modeling.Specifically, to deal with different state/observation spaces in different games, we analogize games as languages by aligning one single game to one single language, thus training different "tokenizers" and a shared transformer for various games.The feature inputs are split according to different entities and tokenized in the same continuous space.Then, two types of transformer-based models are proposed as permutationinvariant architectures to deal with various numbers of entities and capture the attention of different entities.MAGENTA is trained on