Development of a Flexible Mizar Tokenizer and Parser for Information Retrieval System
Kazuhisa Nakasho · Annals of Computer Science and Information Systems · 2019
In this paper, we explain the development of a new Mizar tokenizer and parser program as a component of a search system that works on the Mizar Mathematical Library.The existing Mizar tokenizer and parser can handle only an article as a whole written in the Mizar language, however, the newly developed program can deal with a snippet of a Mizar article.In particular, since it is possible to handle a snippet of an article without specifying a vocabulary section of an environment part, it is expected that user input efforts will be greatly reduced. I. MOTIVATION T HE AUTHOR is developing a new information retrievalsystem that works on the Mizar Mathematical Library (MML) [1].In this paper, we explain a developed tokenizer and parser program of the Mizar language as a component of our search system.