R-D based quantization in H.264
Marta Karczewicz, Peisong Chen, Yan Ye, Rajan L. Joshi · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · 2009
In this paper, a rate-distortion optimized quantization scheme is described with application to H.264 video encoding. An efficient implementation of H.264 macroblock level adaptive quantization parameter selection is also described. Together these two encoder-only changes can achieve on average over 6% bit rate reduction under common testing conditions that are used in the H.264 standardization community. The described techniques provide this improvement in compression capability while retaining conformance of the encoded data to the H.264 standard. Thus, full compatibility with standard decoders can be achieved when applying these techniques.