Stop Taking the Scenic Route: the Shortest Distance Between the CPU and the NIC is MMIO
Wei Siew Liew, Ashfaqur Rahaman Sajib, James McMahon, Ryan Stutsman, Vijay Nagarajan · 2025
What is the fastest way to transfer data from the CPU to a network interface card (NIC)? Conventional wisdom suggests that the answer is direct memory access (DMA), and recent works have advocated for leveraging cache-coherent I/O interconnects. However, in this paper we examine the arguments against memory-mapped I/O (MMIO) and show that high write throughput can be achieved with MMIO by relaxing ordering constraints. We also propose efficient hardware for recovering ordering at the NIC to ensure correctness while taking advantage of the performance benefits of unordered MMIO writes.