Concurrent data streams with heterogeneous devices
Jussi Myllymaki · 1997
The ability to access several datasets concurrently on heterogeneous storage devices is becoming increasingly important for data-intensive applications, including database, data mining, and data visualization systems. The basic problem faced by applications is that while datasets can reside on a variety of storage systems such as secondary, tertiary, and network storage, the CPU can only operate on memory-resident data. Practical solutions are required to allow applications to move datasets from heterogeneous storage devices into memory and back to the devices while maximizing data transfer e ciency and minimizing the amount of time the CPU waits for I/O. Akey factor in achieving high data transfer e ciency is to exploit I/O concurrency. The continually increasing performance gap between CPUs and storage devices has made it imperative for the computer system to perform data transfers on several storage devices concurrently. Operating systems have traditionally attempted to increase I/O concurrency and reduce the amount of time the CPU waits for I/O by overlapping the CPU processing of one application with the I/Os of another (inter-application I/O