Weight sharing on naive bayes document model
Kazumi Saito, R. Nakano · Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005. · 2006
In this paper, we study weight sharing on the naive Bayes document model. Firstly we consider splitting words into a relatively small number of groups such that words in each group have the same parameter value. This problem can be regarded as a probabilistic parameter sharing task. In this task, we formalize the problem in terms of maximum likelihood estimation, and then propose an algorithm for this purpose. Secondly we focus on an adaptive hyperparameter estimation problem based on prior distributions constructed by using such word groups. This problem can be regarded as a hyperparameter sharing task. In this task, we describe a framework and algorithm, which enables to derive the unique optimal solution in the context of leave-one-out cross validation. In our experiments using a benchmark document set called webkb, we show a series of simulation results using the proposed algorithms.