Summary
International Symposium on Nonlinear Theory and its Applications
2009
Session Number:A2L-D
Session:
Number:A2L-D2
Phase Transition of Generalization Errors in Variational Bayes Learning
Shinji Oyama, Sumio Watanabe,
pp.-
Publication Date:2009/10/18
Online ISSN:2188-5079
Summary:
In a mixture of exponential probability distributions, the mean field approximation is used in wide applications because it provides the fast learning algorithm. The mean field approximation in Bayesian learning is called the variational Bayes method. It was clarified by Kazuho Watanabe et. al. that there exists the phase transition of the variational free energy with respect to the hyperparameter. In this paper, we study the generalization errors of variational Bayes learning, and experimentally show the following facts. (1) The generalization error also has the phase transition. (2) At ordinary point, the generalization error strongly depends on the condition that the true distribution is contained in the learning machine or not, whereas, at the critical point, the generalization error does not depend on the condition.