High Performance Computing and I/O Architectures for Database and Knowledge Discovery: The System Design Perspective.
Richard Lundeen, Steve C. Chiu · 2006
Abstract * Research in parallel database (DB) and data mining (DM) algorithms has experienced a significant growth due to advancements in high performance computing (HPC) systems. Enabling technologies such as multi-core processors, object-based storage and high-bandwidth interconnects helped propel innovations to address fast increasing demands in scientific and commercial computing. Large-scale applications involving data in the order of tera-bytes or beyond, not uncommon, require characterization of the HPC system designs to address potential performance bottlenecks. This paper will attempt to design and characterize HPC architectures with novel micro-electromechanical system (MEMS) based storage, where parallel DB and DM algorithms are utilized for inferences and knowledge discovery. A visualization system exploiting parallel message passing interface and open source libraries will be developed. Findings and multiple components from the proposed research may also be extensible to other scientific areas.