System Modeling and Implementation of MPEG-4 Encoder under Fine-Granular-Scalability Framework
Brian L. Evans, Wei Li, Zhenxun Xiao · 2002
Stream media over the Internet is now the mainstream of media communications due to the blooming of the Internet as well as the increasing demanding. A critical issue in providing quality service of media over the Internet is scalability. In this literature survey, we will review the four basic scalable profiles proposed by Moving Picture Experts Group (MPEG), namely data-partitioning, SNR scalability, spatial scalability and temporal scalability. Furthermore, we will investigate the newly proposed amendment of MPEG-4, known as fine-granular-scalability (FGS) framework. We will discuss the relationship between FGS framework and MPEG scalable profiles. The objective of the project is to try to adopt the scalable profiles with the newly proposed FGS framework. The work will be separated into two stages, system modeling and software implementation. Introduction: MPEG-1, MPEG-2 and MPEG-4 are a set of standards developed by moving pictures experts group (MPEG) for compression of motion video optimized at various rates. Standards developed by MPEG only define the syntax of encoded bit streams, and they can be applied to all kinds of applications. Due to the blooming of the Internet and increasing demand, streaming videos over the Internet has become the mainstream of media communications [1, 6]. In this survey, we will first present an overview about the streaming media over the Internet, specifically the scalability issue. Then, we will investigate the scalability profiles in MPEG-2. After that, we will introduce the amendment of fine-granular-scalability framework for MPEG-4 as well as the related open issues. Finally, we will formulate our objective and the way that we are going to approach it. Overview of Streaming Media over the Internet: Providing streaming media services over the Internet poses higher requirements than traditional video coding and decoding techniques, which are designed to provide optimized services for certain video quality at given bit rates [4]. These special requirements are classified as scalable issue, and some constraints for providing scalable service include, 1. be thrifty in using network bandwidth; 2. be able to adapt to the dynamic distribution of bandwidths caused by network congestion; 3. be able to survive on top of best-effort IP service; 4. not pose requirement for extra disk space. MPEG standards adopt a technique called the layered scalability to provide efficient and economical scalable service. The idea is to separate the media information into different layers, with a base-layer containing coarse quality video and enhancement layers containing fine information. By separating information into different layers, there are three obvious advantages, 1. If the media server knows in advance of the users’ processing capabilities and the bandwidth of channels, it can select different layers of information to transmit accordingly; 2. If the media server does not know in advance about the network bandwidth, it can give different priorities to different layers. And the routers on the channel can select the various quality of services on-the-fly; 3. If the media server does not know in advance about the users’ processing capabilities, it can just transmit the layered content to different users, and let the users to choose the encoded information according to their own processing capabilities accordingly. Different Scalable Profiles in MPEG-2: The scalability issue has been extensively investigated in MPEG-2, and there are four basic profiles proposed to provide the scalable service, known as data partitioning, SNR scalability, spatial scalability and temporal scalability [5]. Figure 1. Illustration of different scalable Profiles, data-partitioning, SNR scalability, spatial scalability and temporal scalability, in MPEG-2, with only two layers, known as base-layer and enhancement-layer. I, P and B in figure refer to Iframe, P-frame and B-frame in MPEG standard. BI, BP, EI and EI denote baselayer I-frame, base-layer P-frame, enhancement-layer I-frame and enhancementlayer P-frame in scalable profiles in MPEG-2 respectively. The arrows connecting different frames show the reference relationship between them. As illustrated in Fig. 1a, the profile of data partitioning partitions an encoded single bit stream into base partition and enhancement partition. Base partition contains the lower order DCT coefficients and enhancement partition contains the high-frequency DCT coefficients. The profile of SNR scalability, as shown in Fig. 1b, generates separate bit streams for different layers. The enhancement layer carries the DCT refinement coefficients that can be used to increase the signal-to-noise ration when used together with the DCT coefficients carried in the base layer. And different SNR layers have the same spatial and temporal resolution. The profile of spatial scalability, as shown in Fig. 1c, is to generate separate layers at different resolutions in spatial domain. Base layer and enhancement layer have the same resolution in time domain. The profile of temporal scalability, as shown in Fig, 1d, is to generate separate layers at different resolutions in time domain. Base layer and enhancement layer have the same resolution in spatial domain. In the four scalability profiles described above, the enhancement layer contains the fine information on different domains. When combined with the corresponding base layer for decoding, it can be used to supplement the fine information into the coarse information carried in the base layer. FGS Framework under MPEG-4: In the proposed scalability profiles of MPEG-2, the enhancement layer is encoded through run-level entropy encoding schema. As a consequence of this encoding schema, the decoder needs to decode all the information in the enhancement layer to be able to enhance the base layer or unable to use it at all. Video coding performance analysis of the quality of service provided by the scalability profiles proposed in MPEG-2 in respect of continuously increasing of available bit rate show a staircase effect, (Fig. 2). Figure 2. Illustration of video coding performance [2] For the sake of proving scalable service, fine-granular-scalability (FGS) framework was proposed as an amendment in MPEG-4 [2, 3]. The key issue of FGS framework is the introducing of a new encoding schema, known as bit-plane (Fig. 3), in substitution of runlength entropy encoding schema which is used in MPEG-2 profiles for the encoding enhancement layer. Figure 3. An example of bit-plane encoding schema. In the figure, each column is the 8-bit representation of a DCT coefficient. Traditional runlevel encoding schema is to encode each DCT coefficient individually. In bit-plane schema, all DCT coefficients are arranged as a matrix, each row with the bit from all DCT coefficients at the same significant position are encoded separately. The bit-plane with all zero bits are not coded. As illustrated in Figure 3, the encoding order for bit-plane encoding schema is from the most significant bit (MSB) to the least significant bit (LSB). When enhancement layer is encoded in this way, decoder is able to utilize partial information of the enhancement layer together with the information from base layer for high quality service.