Summary
International Symposium on Nonlinear Theory and its Applications
2009
Session Number:B2L-D
Session:
Number:B2L-D2
Generalization Error by Langevin Equation in Singular Learning Machines
Taruhi Iwagaki, Sumio Watanabe,
pp.-
Publication Date:2009/10/18
Online ISSN:2188-5079
Summary:
The Langevin equation implies an algorithm that can generate samples from the stationary distribution of a biased random walk, which is equivalent to the posterior distribution of Bayesian learning. The Langevin algorithm uses gradient information of the target distribution; therefore it is expected to be more efficient than the Metropolis method especially in wide parameter space of singular learning machines e.g. neural networks. In this paper, we will discuss experimental results of generalization errors of both the Langevin algorithm and the Metropolis method for neural networks with practical dimension.