Optimize the TX Architecture of RDMA NIC for Performance Isolation in the Cloud Environment
Yunkun Liao, Jingya Wu, Wenyan Lü, Xiaowei Li, Guihai Yan · 2023
Remote Direct Memory Access (RDMA) is a promising technology for achieving low latency and high bandwidth access to remote memory. However, performance interference exists when multiple tenants share an RDMA Network Interface Card (RNIC) in the cloud environment. Although some initial studies have investigated the root cause and possible solutions to RDMA performance interference, there is no research to analyze and solve the performance interference from the RNIC architecture. Compared with the existing software approach, optimizing RNIC architecture can introduce less performance and CPU overhead. This paper addresses performance isolation by modeling, analyzing, and optimizing the transmit-side (TX) RNIC architecture. First, we introduce a baseline TX RNIC architecture to explain the existing performance interference. Then, we propose separate caching and slicing execution to avoid the bandwidth-sensitive tenants affecting latency-sensitive tenants. Later, we add isolated backpressure and adaptive Weighted Round-robin scheduling to ensure the bandwidth-sensitive tenants share the bandwidth equally. Our experiments show that these optimizations achieve near-optimal performance isolation.