Saliency Detection From 4D Light Field Images Using an Optimized CNN-based Hybrid Model
Parvathy Prathap, J Jayakumari · 2022 IEEE 19th India Council International Conference (INDICON) · 2022
Light Field Imaging, which captures a 4-dimensional scene representation, has numerous applications across multiple fields. This paper proposes an optimized hybrid saliency detection scheme that incorporates the goodness of handcrafted light field-specific features as well as deep features to create saliency maps out of 4D light field images. An optimized Convolutional Neural Network (CNN) architecture is proposed to extract deep features from light field all-in-focus images. The learning rate, momentum and encoder depth of the proposed CNN are optimized using the Bayesian optimization approach. More adversarial samples are being added to the training set to make it more varied and generalizable. Simulation results indicate that a proper hyperparameter tuning technique can improve the CNN performance and decrease the training time taken as it leads to faster convergence. Manual light field-specific feature extraction is often considered to be more reliable than CNN-based feature maps owing to the inherent ‘black box' nature of convolutional neural network architectures. But saliency maps relying only on manual light field-specific features fail to distinguish salient objects under challenging scenarios. To overcome this, a saliency map refinement step is employed wherein the effectiveness of both the CNN feature-based and manual feature-based saliency maps are combined to form a hybrid saliency map. The proposed optimized 4D hybrid saliency detection model has achieved an average increase of 20.90% and 19.01% in the F-measure and S-measure respectively as compared to the state-of-the-art algorithms considered here.