What do tokens know about their characters and how do they know it?
Ayush Kaushal, Kyle Mahowald · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · 2022
Pre-trained language models (PLMs) that use subword tokenization schemes can succeed at a variety of language tasks that require characterlevel information, despite lacking explicit access to the character composition of tokens.Here, studying a range of models (e.g., GPT-J, BERT, RoBERTa, GloVe), we probe what word pieces encode about character-level information by training classifiers to predict the presence or absence of a particular alphabetical character in a token, based on its embedding (e.g., probing whether the model embedding for "cat" encodes that it contains the character "a").We find that these models robustly encode character-level information and, in general, larger models perform better at the task.We show that these results generalize to characters from non-Latin alphabets (Arabic, Devanagari, and Cyrillic).Then, through a series of experiments and analyses, we investigate the mechanisms through which PLMs acquire English-language character information during training and argue that this knowledge is acquired through multiple phenomena, including a systematic relationship between particular characters and particular parts of speech, as well as natural variability in the tokenization of related strings.