Bringing Context-Aware Completion Suggestions to Arbitrary Text Entry Interfaces
Timothy J. Aveni, Armando Fox, Bjoern Hartmann · 2023
Large language models (LLMs) can predict “obvious” next steps that users will take in text entry fields, especially the tedious components of tasks like software engineering or email composition. These models are not only useful in large, unbroken text fields, however. We present OmniFill, a browser extension that detects text entry fields and offers “autofill”-style suggestions based on context from the browsing session. The system constructs an LLM prompt that includes three main components: (a) a description of the active tab’s text fields and their current values, (b) information from the user’s recent web browsing context, and (c) a history, if available, of the user’s prior submissions to the web form (alongside those submissions’ associated browsing context). Suggestions from the LLM’s response are offered to the user to be automatically typed into each corresponding text field. We offer a motivating example of a time-saving interaction and discuss the broader utility of interface-agnostic LLM integrations.