Proposal of Consistent Learning Model with Exchange Monte Carlo
Hiroki Shibata, Yasufumi Takama · 2022
This paper proposes a consistent learning model based on Exchange Monte Carlo Method. The paper also gives discussion with respect to experiments on the synthesized case. Learning model is currently focusing on the model with the interface for input and output. On that model, preparing a dataset remains in human's work, and there is still not sufficient research how to prepare a valuable dataset efficiently. Exchange Monte Carlo is used widely for both purposes of optimization and estimation of a probability distribution, and it has the ability to combine any probability model in one and sample the model's state efficiency from the combined probability model. From this point of view, when we consider the three models, i.e., real space, learnt model, and model of parameter distribution, we can combine them and construct the consistent model that explains the phenomena of learning consistently. It is supposed that those samples give us valuable dataset and set of the learnt parameter, when the parameter space also modeled with Bayesian inference framework. With these ideas mentioned so far, this paper proposes a generalized consistent probability model of the real space, learning model, and parameters' distribution of the learning model. To challenge to the sampling problem on high-dimensionality of the consistent model, Exchange Monte Carlo and Hamiltonian dynamics are employed. Experiments show the proposed method works on the synthesized case, that is, the original distribution is approximated well and parameter is optimized too, only by sampling from the consistent model without preparing dataset.