My Climate Advisor: An Application of NLP in Climate Adaptation for Agriculture
Vincent A. Nguyen, Sarvnaz Karimi, Willow Hallgren, Ashley Harkin, Mahesh Prakash · 2024
Climate adaptation in the agricultural sector necessitates tools that equip farmers and farm advisors with relevant and trustworthy information to help increase their resilience to climate change.We introduce My Climate Advisor, a question-answering (QA) prototype that synthesises information from different data sources, such as peer-reviewed scientific literature and high-quality, industry-relevant grey literature to generate answers, with references, to a given user's question.Our prototype uses open-source generative models for data privacy and intellectual property protection, and retrieval augmented generation for answer generation, grounding and provenance.While there are standard evaluation metrics for QA systems, no existing evaluation framework suits our LLM-based QA application in the climate adaptation domain.We design an evaluation framework with seven metrics based on the requirements of the domain experts to judge the generated answers from 12 different LLMbased models.Our initial evaluations through a user study via domain experts show promising usability results.