Leveraging Multimodal LLMs for Plant Species Identification and Educational Insights
Yuze Du, Yingjia Wang, Eric Zhao · 2024
In this study, we investigate the potential of multimodal large language models (LLMs) for plant species identification and educational enrichment. Using an annotated dataset focused on fungi, particularly those classified as edible or non-edible, we implement a practical application that allows users to upload plant images. The LLM then identifies the species, determines its edibility, and provides detailed information on its characteristics. For edible species, the model offers culinary insights and preparation methods, while also delivering comprehensive educational content on plant ecology and cultural significance. Our approach showcases the ability of LLMs to bridge image recognition with rich, text-based knowledge, facilitating an interactive learning experience that promotes plant literacy and practical understanding. This study highlights the effectiveness of LLMs in educational tools and their potential to enhance public awareness of plant species, including fungi, through visual and contextual data fusion.