Adaptive Model Selection for Video Super Resolution
Chenge Jia, Jie Ren, Ling Chao Gao, Zhiqiang Li, Jie Hong Zheng, Zheng Wang · 2022
The recent breakthrough of video super-resolution (VSR) techniques has shown impressive results in many computer vision tasks, with deep neural networks as the dominant approach for building VSR. While a wide range of VSR models is available with different processing capabilities and overhead, there is no consensus on the best model to use. This is because the optimal choice changes depending on the input video and the quality requirement. We present AssyVSR, an adaptive framework to dynamically choose the right VSR model to use from a model pool based on the input video and quality requirement. AssyVSR achieves this by using machine learning to build a model selector based on classification. Our classifier takes as input a set of numerical feature values of the input video and then predicts the best model to use for a given video resolution. The model is trained offline from training data, and the trained model can be applied to any unseen videos. AssyVSR further employs transfer learning to quickly update its model selector when the pool of candidate models changes without incurring significant training overhead. We evaluate AssyVSR by applying it to a representative mobile computation offloading environment using real hardware and over 3000 videos. Experimental results show that AssyVSR outperforms the state-of-the-art (SOTA) model selection scheme and the SOTA super-resolution model by producing higher-quality videos and reducing the response time by over 60% when compared with the SOTA super-resolution model.