Fast CSV Loading Using GPUs and RDMA for In-Memory Data Processing
Alexander Kumaigorodski, Clemens Lutz, Volker Markl · Gesellschaft für Informatik (GI) · 2021
Comma-separated values (CSV) is a widely-used format for data exchange. Due to the format's prevalence, virtually all industrial-strength database systems and stream processing frameworks support importing CSV input. However, loading CSV input close to the speed of I/O hardware is challenging. Modern I/O devices such as InfiniBand NICs and NVMe SSDs are capable of sustaining high transfer rates of 100 Gbit/s and higher. At the same time