Smart-Pikachu: Extending Interactivity of Stuffed Animals with Large Language Models
Toma Itagaki, Richard C. Li · 2023
We propose Smart-Pikachu, a stuffed animal equipped with sensing and actuation to explore the use of large language models (LLM’s) with sensor data inputs. The augmentation of pressure sensing will allow for the LLM to interpret various interactions such as hugs and handshakes with the user. Furthermore, the actuation capabilities will extend our system’s interactivity by providing physical feedback to the user. We will also incorporate text-to-speech output from the LLM to add another mode of interaction between the system and user. In this Student Innovation Challenge, we intend to explore applications at the intersection of sensing and interaction through LLM’s and demonstrate an extension of LLMs’ multimodal capabilities.