Manipulation and Measurement of Knowledge Representations of Language Models
Raghav Garg, Mehul Manu, Charu Charu, Neha Singh Chauhan, Naveen Naval, C. Tharini · 2024
Modern neural language models (LMs) are often pre-trained with a self-supervised objective: are given with texts that have piece(s) withheld, and asked to generate the withheld portions of the text. Large amounts of factual knowledge can be stored by language models but it is yet unclear if these models can apply this knowledge to logical reasoning. Tasks that come after may benefit from the factual knowledge that was gathered during pretraining and saved in Language Models’ (LMs’) parameters. The study introduces BART, a technique for learning to translate natural language claims into fact encodings stored in the internal representation system of an LM. BART leverages architecture of denoising autoencoder to refine and manipulate language by pre-training and fine-tuning for effective management of text through utilization of semantic and linguistic knowledge. BART encodings can be employed as knowledge editors to adjust subsequent generation to be compatible with freshly learned facts when applied to LM concealed representations. Because BART encodings reveal which attributes LMs already attribute to the indicated entities, it can also be used as probes when compared to LM representation. In certain situations, this allows for the prediction of when LMs will produce outputs that are at odds with input text or background knowledge. BART provides a link between the work on probing, prompting, and LM editing, as well as a path towards universal tools for fine-grained knowledge control and inspection in LMs. The research gap identified is effective application of knowledge in language models to enhance logical reasoning and maintaining coherence using BART through effective application of acquired knowledge.