Dynamic Session Management Based on Reinforcement Learning in Virtual Server Environment
Kimihiro Mizutani, Izumi Koyanagi, Takuji Tachibana · 2010
Abstract — In a virtualized server environment, machine resources such as CPU and memory are shared by multiple services. In such an environment, as the number of sessions for each service increases, the amount of resources that are utilized by the services increases. If thrashing occurs due to a lack of resources, the performance of the server is degraded. It is effective to estimate the amount of used resources; however, it is hard to estimate the amount of resources that are used dynamically by multiple services. In this paper, we propose a dynamic session management based on reinforcement learning in order to utilize the resources effectively and avoid the thrashing. In the proposed method, a learning agent estimates the amount of used resources from the response time for a service request. Then, the agent decides the acceptance or rejection of an arriving session request with Q-learning. Because this method can be implemented easily in a physical machine, it is expected that our proposed method is used in a real environment. We evaluate the performance of our proposed method with a simulation. From the simulation results, we show that the proposed method can allocate the resources to multiple services effectively while avoiding the thrashing and can perform the priority control for multiple services.