| Presentation | 2012-11-07 Free Energy and Generalization error of Local mimima in Variational Bayes Learning Fumito NAKAMURA, Sumio WATANABE, |
|---|---|
| PDF Download Page |  PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | Variational Bayes learning approximates the posterior distribution with small computational costs, however, it has several local minima which depend on initial values. The variational free energy can be calculated using only training samples, whereas the generalization error not. To compare several local minima, the relation between the variational free energy and the generalization error are necessary. In this paper, we experimentally calculate the free energy and the generalization error for each local minimum, and report the following results. If a true distribution is regular for a statistical model, then the local minimum that minimizes the variational free energy also makes the generalization error minimal. If a true distribution is singular for a statistical model, then the local minimum that minimizes the variational free energy does not always make the generalization error minimal. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | Variational Bayes learning / Variational free energy / generalization error |
| Paper # | IBISML2012-42 |
| Date of Issue |
| Conference Information | |
| Committee | IBISML |
|---|---|
| Conference Date | 2012/10/31(1days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | |
| Chair | |
| Vice Chair | |
| Secretary | |
| Assistant | |
| Paper Information | |
| Registration To | Information-Based Induction Sciences and Machine Learning (IBISML) |
|---|---|
| Language | JPN |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Free Energy and Generalization error of Local mimima in Variational Bayes Learning |
| Sub Title (in English) | |
| Keyword(1) | Variational Bayes learning |
| Keyword(2) | Variational free energy |
| Keyword(3) | generalization error |
| 1st Author's Name | Fumito NAKAMURA |
| 1st Author's Affiliation | Tokyo Institute of Technology, Dept. of Computational Intelligence and Systems Science() |
| 2nd Author's Name | Sumio WATANABE |
| 2nd Author's Affiliation | Tokyo Institute of Technology, Dept. of Computational Intelligence and Systems Science |
| Date | 2012-11-07 |
| Paper # | IBISML2012-42 |
| Volume (vol) | vol.112 |
| Number (no) | 279 |
| Page | pp.pp.- |
| #Pages | 7 |
| Date of Issue | |