ADAPT-T: An Adaptive Algorithm for Auto-Tuning Worker Thread Pool Size in Application Servers
Nilushan Costa, Malith Jayasinghe, Ajantha Sanjeewa Atukorale, Supun Abeysinghe, Srinath Perera, Isuru Perera · 2019
Modern application servers run on multi-core hardware and these servers use multi-threading to achieve high performance. Performance of such servers is highly dependent on the number of processing threads in the worker thread pool. The optimal thread pool size (that would result in the best performance) depends on workload properties (both the application and the incoming arrival pattern). In this paper, we propose ADAPT-T, a lightweight, adaptive algorithm that can auto-adjust the thread pool size to optimize the latency. ADAPT-T exploits the concave upward property of system performance and optimizes the latency on-line by periodically monitoring the server's behaviour and then adjusts the thread pool size based on monitored statistics. It has a special locking mechanism to minimize the frequent fluctuations that may occur in latency while tuning the thread pool size. We perform extensive experiments under different workload patterns and show that ADAPT-T performs well under a wide range of workload scenarios compared to other policies.