A Word Elimination Strategy for Learning Document Representation
Ying Liu, Tongzhou Zhao, Yue Chai, Yiqi Jiang · IOP Conference Series Materials Science and Engineering · 2018
Computing word vectors based on neural network has motivations on document representation. Word elimination can enhance the extract effective of the quality of valuable feature of a document. In this paper, we propose a model named PV-IDF to eliminate redundancy and refine features to improve the performance of the classify model, with which tokens that carry semantic information of a document. The results show that PV-IDF model achieves state-of-art performance, especially for short-length document representation.