Scene graph generation based on global embedding and contextual fusion
zhiyong zhao, Ronglin Hu, hongtai ma, Xinxin Zhang · Third International Conference on Computer Science and Communication Technology (ICCSCT 2022) · 2022
As a structured abstraction method for objects and their interactions in visual scene, scene graph captures entities in the scene and the relationships between the entity pairs, and helps in understanding the visual scene better. Currently, scene graphs in most research works are generated by modeling using context information among targets, focusing only on the inference process but ignoring the integrity of the input target information and the impact of the global information of the targets on relationship inference. Therefore, a new scene graph generation method based on global embedding and contextual fusion (GECF) is proposed in this paper. In this method, richer entity information is obtained by embedding global information into the entity features, while more robust inference of entity interaction information and more reasonable relationship fusion are acquired by combining the attention weighting module and the context inference module as a joint inference module, and merging the obtained entity features according to their discrepancy. The experiment on Visual Genome dataset shows that GECF method performs better than the existing methods in scene graph visual relationship detection.