Bucket Spreading Parallel Hash: A New, Robust, Parallel Hash Join Method for Data Skew in the Super Database Computer (SDC)
Masaru Kitsuregawa, Y. Ogawa · Very Large Data Bases · 1990
The Super Database Computer (SDC) is a highperformance relational database server for a joinintensive environment under development at University of Tokyo. SDC is designed to execute a join in a highly parallel way. Compared to other join algorithms, a hash-based algorithm is quite efficient and easily parallelieed, and has been employed by many database machines. However, in the presence of data skew, it’s hard to distribute load equally among processing modules (PMs) by statically allocating buckets to PMs, as in the conventional parallelieing strategy. Thus, performance is severly degraded. In this paper, we propose a new parallel hash join method, the bucket spreading strategy, which is robust for data skew. During partitioning relations, each bucket is again divided into fragments of the same sise and these fragments are temporarily placed on PMs one by one. Then each bucket is dynamically allocated to a PM which actually carries out the join of the bucket, and all fragments of the bucket are collected in the corresponding PM. In this way, the bucket spreading strategy evenly distributes the load among the PMs and parallelism is always fully exploited. The architecture of SDC is designed to support the bucket spreading strategy; a mechanism which distributes the buckets flatly among the PMs is embedded in the hardware of the interconnection network. Simulation results confirm that the bucket spreading strategy is robust for data skew and attains very good scalability. Permission to copy without fee all or part of this material is granted provided that the copies are not made or distributed for direct commercial advantage. the VLDB copyright notice and the title of the publication and its date appear. and notice is given that copying is by permission of the Very Large Data Base Endowment. To copy otherwise. or to rcpuhlish. requires a kc and/or special permission from the Endowment. Proceedings of the 16th VLDB Conference Brisbane, Australia 1990 Yasushi Ogawa Research and Development Center, RICOH Co., Ltd. 16-l Shinei-cho, Kohoku-ku,