Design and optimization of architectures for data intensive computing
Alok Choudhary, Jayaprakash Pisharath · 2005
Computer technology in recent years is propelled by new hardware designs, advanced software features and multitudinous user demands. Due to this, the data collected and managed by applications is also abundant. Today's connect anytime and anywhere society based on the use of advanced digital technologies has increased the expectations of users. Data access is expected to be quick, highly reliable and fast. Future systems are expected to even more data centric and compute intensive. This fact is used as a motivation in this research work to propose new techniques and optimizations that enable high speed access to data. A three-tier data driven approach is taken to propose new architectural designs and techniques. The three perspectives of data that is used are streaming data, structured databases and new-age massive datasets (could be structured or unstructured). Streaming data is widely popular and is still emerging. In this work, new scheduling and resource allocation strategies are proposed for such stream data systems. Both performance and energy improve dramatically when the proposed schemes are deployed in existing heterogeneous systems. Also, a new framework of analysis is proposed to comprehensively study the performance and energy consumption of such high performance systems. The next focus area of this work is modern database systems. Storage technology has evolved significantly in the recent years. In this study, modern memory technology is used as a motivation to tune and adapt modern DBMS to modern storage technology. First, dynamic hardware management schemes are proposed. Secondly, the query optimizer is also modified to reflect the change in the storage architectures. Other emerging storage paradigms are also considered in this work. With data growing at alarming proportions, data mining is emerging to be an excellent tool to automatically extract useful information from such large datasets. The growth of data, the advancements in data mining tools, and the user expectations are to tally incongruent to the improvements in the performance of general purpose computing systems. To alleviate this, a new data mining system is designed that achieves massive speeds that can never be achieved using traditional high performance techniques.