Design of a Q-learning-based client quality selection algorithm for HTTP adaptive video streaming
Maxim Claeys, Steven Latré, Jeroen Famaey, Tingyao Wu, Werner Van Leekwijck, Filip De Turck · Ghent University Academic Bibliography (Ghent University) · 2013
Over the past decades, the importance of multimedia services such as video streaming has increased considerably.HTTP Adaptive Streaming (HAS) is becoming the de-facto standard for adaptive video streaming services.In HAS, a video is split into multiple segments and encoded at multiple quality levels.State-of-the-art HAS clients employ deterministic heuristics to dynamically adapt the requested quality level based on the perceived network and device conditions.Current HAS client heuristics are however hardwired to fit specific network configurations, making them less flexible to fit a vast range of settings.In this article, an adaptive Q-Learning-based HAS client is proposed.In contrast to existing heuristics, the proposed HAS client dynamically learns the optimal behavior corresponding to the current network environment.Considering multiple aspects of video quality, a tunable reward function has been constructed, giving the opportunity to focus on different aspects of the Quality of Experience, the quality as perceived by the end-user.The proposed HAS client has been thoroughly evaluated using a network-based simulator, investigating multiple reward configurations and Reinforcement Learning specific settings.The evaluations show that the proposed client can outperform standard HAS in the evaluated networking environments.