LLM-aided Knowledge Graph construction for Zero-Shot Visual Object State Classification
Filippos Gouidis, Katerina Papantoniou, Konstantinos E. Papoutsakis, Theodore Patkos, Antonis Argyros, Dimitris Plexousakis · 2024
The problem of classifying the states of objects using visual information holds great importance in both applied and theoretical contexts. This work focuses on the special case of Zero-shot Object-Agnostic State Classification (ZS-OaSC). To tackle this problem, we introduce an innovative strategy that capitalizes on the capabilities of Graph Neural Networks to learn to project semantic embeddings into visual space and on the potential of Large Language Models (LLMs) to provide rich content for constructing Knowledge Graphs (KGs). Through a comprehensive ablation study, we explore the synergies between LLMs and KGs, uncovering critical insights about their integration in the context of the ZS-OSC problem. Our proposed methodology is rigorously evaluated against current state-of-the-art (SoA) methods, demonstrating superior performance in various image datasets.