Utility of Local Quantized Large Language Models in Semantic Navigation
Arturs Elksnis, Ziheng Xue, Feng Chen, Ning Wang · 2024
The use of Large Language Models (LLMs) in Semantic Navigation (SN) has so far centered around high-parameter count cloud-hosted LLMs. We wanted to explore the capabilities of low parameter-count quantized local LLMs that can be hosted on a consumer grade laptop. We use such LLMs in extracting semantic knowledge from the environment. This knowledge is then used to create an executable path which leads the agent to a goal set by human operator. We explore two main tasks for which LLM can be employed: room classification and goal selection and compare the performances of several LLMs and appropriately constructed Support Vector Classifier (SVC) for this purpose. Finally we present a simple framework for using LLM in SN which we used for evaluation purposes. We compare 4 LLMs - Gemma, Llama3 and two quantizations of Mistral. Among the LLMs we tested, Llama3 is the most accurate room classifier and combined with its goal selection ability it lends itself well to SN.