Portal Framework for Computation within the eMinerals Project
R. P. Tyer, Mark Calleja, Richard P. Bruin, Christopher A. R. Chapman, MT Dove, RJ Allan · UCL Discovery (University College London) · 2004
This paper describes the eMinerals minigrid infrastructure and motivates the need for a portal based environment as an interface to this infrastructure. The paper then describes the ComputePortal developed by the project as a portal interface to the compute resources within the minigrid. Its architecture, functionality and integration with other portals in use by the project are discussed. Finally an example use case is outlined which shows the potential functionality available via an integrated portal environment. 1. eMinerals Project The ‘Environment from the molecular level’ (eMinerals) project [1] is a NERC funded eScience pilot project focused on fundamental science problems associated with key environmental issues such as nuclear waste storage and pollution. The project includes collaborators from Bath University, Cambridge University, CCLRC, the Royal Institution, UCL and Reading University. The scientific aims of the project are primarily focused on the use of computer simulations performed on systems with molecular length and time scales in order to gain insight into key environmental issues. Aside from the scientific issues, this research is challenging both in terms of the computational power required to tackle realistic system sizes with the required accuracy and the data management issues associated with handling large amounts of data over a distributed virtual organisation. The computational requirements of the project vary significantly depending primarily on both the simulation code and physical system in question but both HPC and HTPC resource are required. Hence the use of eScience and Grid technologies to develop an integrated infrastructure and working environment provides enticing opportunities to facilitate and enhance the scientific work performed by the project. 2. eMinerals Minigrid The project has investigated and deployed a number of different pieces of eScience technology in order to build the eMinerals Minigrid [2], an integrated grid infrastructure combining both compute and data resources. Along with the physical resources, the minigrid includes a suite of client tools and portals which act as brokers to the compute and data servers. A schematic view of the minigrid infrastructure is presented in Figure 1. 2.1 Compute Resources The compute infrastructure used by the project includes both HPC and HTPC resources as both are required by different computational codes and classes of problems. The HPC resources on the minigrid include three 16 node Beowulf cluster machines and a 24 node IBM pSeries parallel computer. In addition the project makes use of other HPC facilities such as the NGS machines and HPCx. These machines run different batch queuing systems and have been integrated into the minigrid using a mixture of Globus Toolkit 2 and 3 [3]. The project has also made heavy use of Condor [4] for HTPC calculations and has been central to the creation of a large (currently around 950 node) Condor pool at UCL. In addition, there is a smaller, but extremely heterogeneous, Condor pool at the Cambridge node. Again Globus Toolkit 2 has been used to integrate these resources in the grid infrastructure.