Investigating OS/DB co-design with SharedDB and Barrelfish
Zaheer Chothia · Repository for Publications and Research Data (ETH Zurich) · 2013
Databases and operating systems both operate over the same hardware but they have differing views of how the resources should be managed.As a result databases forgo the system services and implement their own scheduling and memory management which have been refined over several decades.Co-design is an area of research to address this by improving the collaboration between both systems.The idea is to integrate application knowledge in the operating system's decisions, and for the database to receive notifications and adapt to changes in system state.This thesis presents work in this area which builds upon two research systems.One is Barrelfish OS, a realisation of the multikernel model which treats the machine as a distributed system.The other is SharedDB, an in-memory relational database designed around sharing of computation which delivers predictable performance on large and complex workloads.Our experimental results on Linux show that less than a quarter of the cores are needed to achieve peak performance when running a TPC-W workload.We are interested in resource consolidation and present an approach for utilising application knowledge in spatial and temporal scheduling of operators.We ported SharedDB to run on top of Barrelfish and conclude with discussion that the combined system makes a good foundation for testing new ideas and optimisations which bisect several layers of the system stack.i 2. Background due to contention.Overcoming this has involved considerable effort to replace with more fine-grained sharing.Another point in the design space is the microkernel [23] which provides a minimal set of mechanisms in the kernel and traditional services running in user-space which implement a specific policy.The main OS abstractions in such a system are address spaces, threads and inter-process communication.Exokernel [12] took this idea further and separates resource protection from management.The kernel exposes just a thin interface over the hardware and the conventional abstractions are implemented in an untrusted library OS.More recently there have also been projects such as Tornado, K42 [13,18] and Corey [6] which target multicore hardware.They demonstrate the benefits of avoiding sharing across cores through careful design of data structures. x86 x64 ARM