Enabling Grids for GATE Monte-Carlo Radiation Therapy Simulations with the GATE-Lab
Sorina Camarasu-Pop, Tristan Glatard, Hugues Benoit-Cattin, David Sarrut · InTech eBooks · 2011
Among radiation therapy simulation methods, Monte-Carlo approaches are known to be the most accurate but they are heavy to use because of their computing time.Nowadays they can be accelerated with the help of the ever-increasing computing power and distributed resources mutualised in clusters, clouds or grids (Montagnat et al. (2005)).Grid infrastructures are used both for experimental and production purposes.They have been designed to support data and computing requirements for a large spectrum of applications from various scientific domains.Nevertheless they are not yet at a "plug and play" phase that would allow applications to be easily and efficiently deployed on the existing infrastructure.Efforts are required to achieve reliable and efficient execution on the grid for new applications and to provide user-friendly execution environments to end-users.This chapter presents a solution for reliable, user-friendly and fast execution of GATE (Jan et al. (2004)) on a grid.Developed within the OpenGate 1 international collaboration, GATE is a Monte-Carlo based open-source software for nuclear medicine simulations, especially for TEP and SPECT imaging, and for radiation therapy applications.The solution proposed here enables transparent grid execution from a user-friendly interface.The application is parallelized automatically and a dynamic partitioning can also be used for further reducing the execution time and improving the robustness to job failures (Camarasu-Pop et al. (2010)).This chapter will discuss in more detail the implemented solution by describing the user interface, the system architecture, as well as the dynamic optimization strategy.Usage and performance results illustrating the system adoption will then be presented.To conclude with, it will discuss the lessons learned in building the system. Related workDifferent parallelization methods for Monte-Carlo simulations have been proposed for execution on distributed environments.This section will present related work on 1 http://opengatecollaboration.healthgrid.org/ Enabling Grids for GATE Monte-Carlo RadiationTherapy Simulations with the GATE-Lab 3 www.intechopen.comparallelization methods, on challenges raised by distributed environments (e.g.production grids), as well as on end user interfaces allowing to run such parallelized applications on grids. Parallelization methodsThe simulation in a particle tracking Monte-Carlo system consists in the successive stochastic tracking through matter of a large number of individual particles.Each particle has an initial set of properties (type, location, direction, energy, etc) and its interaction with matter is determined according to realistic interaction probabilities and angular distributions.Accurate results require the simulation of a large number of particles and physical interactions can also produce other particles that must be tracked.Therefore, typical radiation therapy simulations can take several days or even weeks to complete on a single computer.However, they can be easily parallelized on distributed systems.Instead of sequentially simulating a large number (up to several billions) of particles, smaller groups (bags) of particles can be simulated independently and eventually merged.This process is valid only if the sub-simulations are statistically independent, which in GATE is guaranteed by using a special random number generator as explained by Reuillon et al. (2008).Among existing parallelization methods, the simplest and most commonly used is particle parallelism.In this case the geometry information is replicated on each processor and particles are distributed between available processors as described in Maigne et al. (2004).The authors present first results obtained by running GATE in parallel on multiple processors of the DataGrid 2 project.In this example the number of particles simulated on each of the N processors represents a fraction P/N of the total of P particles.This static distribution of particles often underexploits resources when the simulation is executed on heterogenous platforms like computer grids.Indeed, as particles are evenly distributed among tasks, tasks running on fast resources complete before other ones and these resources may remain idle if the scheduler cannot assign them other tasks (e.g. at the end of the simulation).Moreover, it may happen that a task is allocated to a slow resource towards the end of the simulation, thus slowing down the completion of the whole application (Cirne et al. (2007)).A possible solution to this problem is the dynamic distribution and/or reassignment of particles to available processors during runtime.Dynamic partitioning is proposed in Galyuk et al. (2002) andin Procassini et al. (2005) for spatial parallelism.Spatial parallelism involves splitting the geometry into domains and then assigning a specific domain to one processor.This method is usually needed when the problem geometry has a significant size so that one processor does not have enough memory to store all particles/zones.Spatial parallelism may introduce load imbalance between processors, as spatial domains will require different amounts of computational work.In Procassini et al. (2005), a dynamic load balancing algorithm distributes the available processors to the spatial domains.Communications are generated between processors to transmit changes from the last state.In Galyuk et al. ( 2002), the parallelization is done using an MPI implementation and is based on a semaphore principle under distributed memory conditions.These two implementations are therefore cluster oriented and are not adapted for grid usage where communications between processors are very costly.Camarasu-Pop et al. (2010) propose a dynamic partitioning algorithm for GATE radiotherapy simulations using pilot jobs.Statistically independent simulations are launched on available resources and they keep on running until the desired number of particles is reached.