Bitrate Adaptation and Guidance With Meta Reinforcement Learning
Abdelhak Bentaleb, May Li Lim, Mehmet N. Akcay, Ali C. Begen, Roger Zimmermann · IEEE Transactions on Mobile Computing · 2024
Adaptive bitrate (ABR) schemes enable streaming clients to adapt to time-varying network/device conditions for a stall-free viewing experience. Most ABR schemes use manually tuned heuristics or learning-based methods. Heuristics are easy to implement but do not always perform well, whereas learning-based methods generally perform well but are difficult to deploy on low-resource devices. To make the most out of both worlds, we earlier developedAhaggar, a learning-based scheme executing on the server side that provides quality-aware bitrate guidance to streaming clients running their own heuristics.Ahaggar's novelty is the meta reinforcement learning approach taking network conditions, clients' statuses and device resolutions, and streamed content as input features to perform bitrate guidance.Ahaggaruses the new Common Media Client/Server Data (CMCD/SD) protocols to exchange the necessary metadata between the servers and clients. WhileAhaggarwas a significant step forward, in this study, we focus on three open areas, namely, ($i$) exploring the performance ofAhaggarin a heterogeneous environment including bothAhaggarand non-Ahaggarclients with varied network conditions and device resolutions, and ($ii$) quantifying the impact of device resolutions on QoE withAhaggar. We thoroughly investigate these areas and report our findings. We also ($iii$) discuss theAhaggardesign choices. Experiments on an open-source system show thatAhaggaradapts to unseen conditions fast and outperforms its competitors in several viewer experience metrics.