Towards a Probabilistic Taxonomy of Many Concepts
Haixun Wang, Hongsong Li · 2011
Knowledge is indispensable to understanding. The ongoing information explosion highlights the need to enable machines to better understand electronic text in natural human language. The challenge lies in how to transfer human knowledge to machines. Much work has been devoted to creating universal ontologies for this purpose. However, none of the existing ontologies has the necessary depth and breadth to offer “universal understanding.” In this paper, we present a universal, probabilistic ontology that is more comprehensive than any of the existing ontologies. It contains 2.7 million concepts harnessed automatically from a corpus of 1.68 billion web pages and two years’ worth of search log data. Unlike traditional knowledge bases that treat knowledge as black and white, it enables probabilistic interpretations of the information it contains. The probabilistic nature then enables it to incorporate heterogeneous information in a natural way. We present details of how the core ontology is constructed, and how it models knowledge’s inherent uncertainty, ambiguity, and inconsistency. We also discuss potential applications, e.g., understanding user intent, that can benefit from the taxonomy.