An integrated learning and approximation scheme for coding of static or dynamic light fields based on hybrid Tucker–Karhunen–Loève transform‐singular value decomposition via tensor double sketching
Joshitha Ravishankar, Mansi Sharma · IET Signal Processing · 2022
Abstract This study presents a scheme for efficient representation, coding and streaming of static or dynamic light fields using the authors’ novel hybrid Tucker‐TensorSketch Karhunen–Loève transform‐singular value decomposition via double sketching (HTTS‐KLTSVD‐DS) algorithm. A deep learning model is employed to obtain acquired images from the light fields by simulating coded aperture patterns. These acquired images can represent the entire light field and are low‐rank approximated using HTTS‐KLTSVD‐DS. Incorporation of double sketching using TensorSketch allows the authors’ algorithm to work faster in a single pass itself and there is no need to store large Kronecker products of Tucker decomposition in the memory. This provides an efficient transmission and streaming adaptability of the light field, making it suitable for 3D display applications. Besides, compact representation of factor matrices by KLT‐SVD in the authors’ proposed model acts as an optimal transform with good energy compaction property. Encoding of low‐rank approximated acquired images using HEVC eliminates intra‐frame, inter‐frame and other intrinsic redundancies in the light field. The authors’ complete light field processing pipeline flexibly works for multiple bitrates and is adaptable for a variety of multi‐view autostereoscopic platforms. Comparison with state‐of‐the‐art codecs shows reasonable savings and PSNR gains for low and high bitrates, while maintaining good reconstruction quality.