Towards a two-phase unsupervised system for cybersecurity concepts extraction
Zhifeng Xiao · 2017
This paper explores a novel named entity recognition approach to locate and classify cybersecurity concepts from unstructured texts. The proposed system follows a two-phase procedure. Phase one aims for named entity location, which can be achieved by any existing NER system that is well trained on generic and annotated English articles. The output of phase one is a processed text in which named entities are located and well marked. In phase two, we prepare two core components that define the domain knowledge. One is a word2vec-based domain model trained on a large corpus of cybersecurity articles, and the other is a domain ontology that comprises hierarchical concept classes as well as instances. With these two components and the output from phase one, we propose a voting-based model to classify the marked named entities into fine-grained classes. The proposed NER system is unsupervised, meaning that no annotated training corpus is needed. As such, it can be easily customized to suit a variety of domains in addition to cybersecurity. Our evaluation shows promising results which indicate the potential of the proposed system.