Scalable parallel database technology
Yu-Lung Lo, Kein A. Hua · Journal of International Crisis and Risk Communication Research · 1996
Although parallel computers were originally designed mostly for scientific and engineering community, it is the emergence of parallel database technologies in recent year popularizing the commercial multicomputer and multiprocessor systems. With the new technologies, it is possible to build high-performance database servers at a much lower cost than equivalent mainframe computers. These systems, however, have to deal with data distributed across many processing nodes. Design issues such as data fragmentation, data allocation and parallel query processing techniques, therefore, are essential to the scalability of these systems. Today, data are allocated in these systems using horizontal partitioning strategies. This approach has a number of drawbacks. These problems are addressed in this thesis using a new multidimensional data partitioning strategy. For query processing, this thesis examines a query optimization technique which includes the cost of load balancing as a new factor for query optimization. Two scheduling techniques, Competition-Based and Planning-Based, are also investigated as they are designed to support the optimizer-assisted load balancing technique. Extensive simulations have been done to study the performance of these techniques. They show that the new schemes provide very significant performance improvement over existing methods. To demonstrate the feasibility of the proposed schemes. A parallel database management system is prototyped on a 64-processor nCUBE/2 computer.