Class‐aware single image to 3D object translational autoencoder
Ceren Güzel Turhan, Hasan Şakir Bılge · IET Image Processing · 2020
The performances of generative adversarial network (GAN) and autoencoder (AE) models on images have been gathering a great deal of interest in terms of transferring them to three‐dimensional (3D) domain. In this study, single image to object reconstruction problem was focused by presenting a novel 2D‐to‐3D AE model inspired by the recent improvements. To benefit from middle‐level features, a model with skip connections was constructed by transferring 2D features to 3D domain. Moreover, the authors considered class‐awareness for obtaining a category‐agnostic model using limited class‐annotations. Apart from recent 3D reconstruction models, they adapted the intersection‐over‐union score based objective, which is used in the object segmentation model, for improving reconstruction performance. With all these contributions, they call their model as skipped volumetric class‐aware AE (SkipVCAE). According to experimental studies, proposed model obtained higher scores than the state‐of‐the‐art model given. The results have proven its performance as a category‐specific and category‐agnostic model together owing to its class‐aware nature. In the further analysis, it was seen that presented model yielded satisfactory results on a single image to object modelling compared to its multi‐view version thanks to class‐awareness.