RMASBench: Benchmarking Dynamic Multi-Agent Coordination in Urban Search and Rescue (Extended Abstract)
Alexander Kleiner, Alessandro Farinelli, Sarvapali D. Ramchurn, Bing Shi, Fabio Maffioletti, Riccardo Reffato · 2013
The development of benchmarking platforms for agent-based sys-tems is crucial to evaluate the performance of state-of-the-art algo-rithms and mechanisms in controlled settings and then determinethe best ones to use under a wide range of realistic conditions.To this end, these platforms need to pose realistic challenges thatmimic those of the real world and at the same time should allowfor easy implementation of new algorithms without requiring re-searchers to implement low-level elements irrelevant to their algo-rithm. To date, very few benchmarking platforms have been de-veloped according to such principles (and adopted by the artificialintelligence community). In particular, we note the dearth of re-alistic testbeds for agent-based coordination mechanisms such asdistributed constraints optimization (DCOP), task allocation (TA),and coalition formation (CF). Those testbeds that do aim to evalu-ate such algorithms (e.g., DCOPolis and CATS) typically provideinputs that are drawn from fixed distributions or define coordinationproblems that are usually static or require significant extensions tocreate dynamic settings and large-scale problems. As a result, thereare currently no well defined realistic benchmarks for DCOP, TA,and CF.Against this background, we develop and evaluate a novel testbedfor multi-agent coordination algorithms called RMASBench. Ourwork builds upon the existing RoboCup Rescue simulation plat-form (RSP) that simulates an urban search and rescue scenario andhas also been used by emergency responders and planners to bothtrain and plan for emergencies. In the RSP, agent designers haveto code police agents to unblock roads, fire brigade agents to ex-tinguish fires, and ambulance agents to rescue trapped civilians.More importantly, the problems that the RSP requires agent de-signers to solve for, are coalition formation (e.g., forming teams ofambulances or fire brigades to save civilians and extinguish fires re-Appears in: Proc. of the 12th Int. Conf. on Autonomous AgentsandMultiagentSystems(AAMAS2013)