Why Files If You Have a DBMS?
Lam-Duy Nguyen, Viktor Leis · 2024
Most Database Management Systems (DBMSs) sup-port arbitrary-sized objects through the Binary Large OBjects (BLOBs) data type. Nevertheless, application developers usually store large objects in file systems and only manage the metadata and file paths through the DBMS. This combined approach has major downsides, including a lack of transactional and indexing capabilities. Two factors contribute to the rare use of database BLOBs: the inefficiency of DBMSs in such workloads and the interoperability difficulties when interacting with external programs that expect files. To address the former, we present a new BLOB allocation and logging design that exhibits lower write amplification, reduces WAL checkpointing frequency, and consumes less storage than the conventional strategies. Our approach flushes each BLOB only once and features only a single indirection layer. Moreover, using the Filesystem in Userspace framework, BLOBs can be exposed as read-only files, allowing unmodified applications to directly access database BLOBs. The experimental results show that our design outperforms both file systems and DBMSs in handling large objects.