Effective Racing on Partially Observable Tracks: Indirectly Coupling Anticipatory Egocentric Sensors With Motor Commands
Martin V. Butz, Matthias J. Linhardt, Thies D. Lonneker · IEEE Transactions on Computational Intelligence and AI in Games · 2010
The TORCS-based Simulated Car Racing Championship (SCRC) poses a demanding challenge for designing an effective racing car controller. Controllers do not receive any global track information, but only perceive simulated, car-centered sensory information about the current, local track properties and about surrounding opponents. Our racing controller, termed COgnitive BOdySpaces: TORCS-based Adaptive Racing (COBOSTAR), uses the sensors that give the most anticipatory information to learn a sensory-to-motor policy, which was optimized by means of the covariance matrix adaptation evolution strategy (CMA-ES). The basic policy was extended by additional modules to prevent detrimental skidding, to safely land after jumps, to implement effective opponent avoidance, and to recover and learn online from accidents. Owing to this approach, COBOSTAR won two out of the three competition legs of the 2009 SCRC and, without hardly any further modifications, also was among the first in the 2010 SCRC despite the addition of noise to the utilized sensors. This paper describes the COBOSTAR controller as it was submitted to the last of the three competition legs in 2009. Evaluations of distinct controller modules are provided where possible. A future outlook summarizes the lessons learned during the design of the racer and proposes the utilization of the framework also in broader research and application contexts.