Multi-Agent Deep Reinforcement Learning for Radiation Localization
Benjamin Totten · 2023
For the safety of both equipment and human life, it is important to identify the location of orphaned radioactive material as quickly and accurately as possible. There are many factors that make radiation localization a challenging task, such as low gamma radiation signal strength and the need to search in unknown environments without prior information. The inverse-square relationship between the intensity of radiation and the source location, the probabilistic nature of nuclear decay and gamma ray detection, and the pervasive presence of naturally occurring environmental radiation complicates localization tasks. The presence of obstructions in complex environments can further attenuate the signal from radioactive material. Current existing localization methods such as those seen by Anderson et al. (2022) that use data fusion, stationary node placement optimization, or path planning solutions rely on pre-existing knowledge of the environment, reducing flexibility. Generalizable localization methods that can localize low-signal targets are needed. In order to create these methods, the research community has shown an increasing interest in self-teaching autonomous solutions, primarily focused on the application of single-searcher architectures that use a branch of machine learning called reinforcement learning, and more recently, multi-agent reinforcement learning. Current single-agent reinforcement learning solutions for localization tasks include a double deep Q-learning approach seen by Liu et al. (2019), an actor-critic method augmented with a particle filter embedded in a neural network by Proctor et al. (2021), and a deep Q-learning method that is also augmented by a particle filter by Zhao et al. (2022). The inclusion of multiple learning agents, however, adds additional complexity and challenges, and multi-agent frameworks cannot be effectively directly extended from systems designed to be single-agent algorithms for localization tasks due to "scalability", "environment non-stationarity", and "credit assignment" challenges. Current multi-agent approaches include architectures by Alagha et al. (2022, 2023) that use centralized training decentralized execution actor-critic methods and unique learning paradigms, such as demonstration cloning. Current multi-agent methods, while effective, fail to address complex environments where signal-blocking obstructions and signal noise are present, and require a significant increase in training time when compared to single-agent architectures. In this work, I present RAD-TEAM, a multi-agent deep reinforcement learning approach to radiation localization. RAD-TEAM is an on-policy model-free policy gradient deep reinforcement learning framework that supports an arbitrary number of agents that autonomously coordinate their efforts in order to find a source of nuclear radiation with an unknown location. Agents use proximity sensors to detect