Extracting structured knowledge for semantic web by mining Wikipedia
Kotaro Nakayama · 2008
Since Wikipedia has become a huge scale database storing wide-range of human knowledge, it is a promising corpus for knowledge extraction. A considerable number of researches on Wikipedia mining have been conducted and the fact that Wikipedia is an invaluable corpus has been confirmed. Wikipedia’s impressive characteristics are not limited to the scale, but also include the dense link structure, URI for word sense disambiguation, well structured Infoboxes, and the category tree. One of the popular approaches in Wikipedia Mining is to use Wikipedia’s category tree as an ontology and a number of researchers proved that Wikipedia’s categories are promising resources for ontology construction by showing significant results. In this work, we try to prove the capability of Wikipedia as a corpus for knowledge extraction and how it works in the Semantic Web environment. We show two achievements; Wikipedia Thesaurus, a huge scale association thesaurus by mining the Wikipedia’s link structure, and Wikipedia Ontology, a Web ontology extracted by mining Wikipedia articles. 1. WIKIPEDIA THESAURUS WikiRelate [3] is one of the pioneers in this research area. The algorithm finds the shortest path between categories which the concepts belong to in a category tree. As a measurement method for two given concepts, it works well. However, it is impossible to extract all related terms for all concepts because we have to search all combinations of category pairs of all concept pairs (2 million × 2 million). Therefore, in our previous research, we proposed pfibf (Path Frequency- Inversed Backward Link Frequency) 1, a scalable association thesaurus construction method to measure relatedness among concepts in Wikipedia. The basic strategy of pfibf is quite simple. The relativity between two articles vi and vj is assumed to be strongly affected by the following two factors: • the number of paths from article vi to vj, • the length of each path from article vi to vj. The relativity is strong if there are many paths (sharing of many intermediate articles) between two articles. In addition, the relativity is affected by the path length. In other 1 The method name was lfibf in the past and was changed to pfibf