On Spatial Reasoning and Perspective Transformation in Language Models
Jason Weston, Antoine Bordes, Sumit Chopra, Alexander M. Rush, Bart van Merriënboer, Armand Joulin, Tomáš Mikolov · arXiv (Cornell University) · 2015
In this paper, we investigate the performance of language models on spatial reasoning with textually presented data, with particular focus on the ability to switch between different perspectives (route vs. survey) - an important component of human spatial reasoning. Our methodology involves the construction of large, automatically labelled corpora, as opposed to crowd-sourced, human-annotated datasets; this approach focuses attention on the language models' command of underlying geometrical principles, rather than their access to background knowledge and commonsense rules-of-thumb. Results reveal that language models can acquire basic spatial reasoning ability after finetuning on targeted tasks, though they do not fully capture the underlying rules. Furthermore, language models show significant differences from humans in the perspective-transformation task, exhibiting distinct patterns of performance across perspective conditions.