Afc: Asymmetrical Feature Coding for Multi-Task Machine Intelligence
Yuan Zhang, Hanming Wang, Yunlong Li, Lu Yu · 2024
In light of the unprecedented success of Artificial Intelligence (AI), the amount of video content intended for machine vision has surpassed that intended for human vision. Therefore, it is crucial to develop customized codecs that are more specialized in machine vision applications. To facilitate the study of this topic, Moving Picture Experts Group (MPEG) has established two working groups, Video Coding for Machines (VCM) and Feature Coding for Machines (FCM). Unlike traditional video coding standards, the output of VCM decoder are fed into machine vision models (in many cases neural networks) instead of being viewed by humans. Thus, the codec is optimized towards high machine task performance rather than high fidelity. FCM takes a further step by directly compressing the intermediate feature tensors of the task neural networks, enabling a balance between compression efficiency and multi-task accuracy. Furthermore, FCM facilitates load balancing, allowing for the realization of Collaborative Intelligence (CI). We propose an advanced FCM algorithm, Asymmetrical Feature Coding (AFC), with novel feature re-duction and feature restoration modules. We evaluated the AFC on multiple datasets under three task scenarios, including object detection, instance segmentation, and object tracking. AFC outperforms the state-of-the-art video and feature compression technologies, achieving an overall of 94.65% BD-rate gain. AFC ranks 1st in the MPEG FCM Call for Proposals (CfP) responses evaluation.