Non-negative Matrix Factorization for Discrete Data with Hierarchical Side-Information
Changwei Hu, Piyush Rai, Lawrence Carin · International Conference on Artificial Intelligence and Statistics · 2016
We present a probabilistic framework for ecient non-negative matrix factorization of discrete (count/binary) data with sideinformation. The side-information is given as a multi-level structure, taxonomy, or ontology, with nodes at each level being categorical-valued observations. For example, when modeling documents with a twolevel side-information (documents being at level-zero), level-one may represent (one or more) authors associated with each document and level-two may represent aliations of each author. The model easily generalizes to more than two levels (or taxonomy/ontology of arbitrary depth). Our model can learn embeddings of entities present at each level in the data/sideinformation hierarchy (e.g., documents, authors, aliations, in the previous example), with appropriate sharing of information across levels. The model also enjoys full local conjugacy, facilitating ecient Gibbs sampling for model inference. Inference cost scales in the number of non-zero entries in the data matrix, which is especially appealing for real-world massive but sparse matrices. We demonstrate the eectiveness of the model on several real-world data sets.