| Presentation | 2009-03-11 On a Relation between a Limit Theorem in Learning Theory and Singular Fluctuation Sumio WATANABE, |
|---|---|
| PDF Download Page |  PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | Let B_g, B_t, G_g, G_t, and V be the Bayes generalization error, the Bayes training error, the Gibbs generalization error, the Gibbs training error, and the functional variance, respectively. If the α posteriori distribution with the inverse temperature β>0 is employed, then the equations of states in learning, E[B_g]=E[B_t]+βE[V] and E[G_g]=E[G_t]+βE[V], hold. Hence B_t+βV and G_t+βV can be applied to model selection and hyperparameter optimization. In the previous papers, we proved these equations on the assumption that the true distribution is contained in the learning machine and that Fisher information matrix is singular. In this paper, we prove the same equations hold on the assumption that the true distribution is not contained in the learning machine and that Fisher information matrix is positive definite. Also we show nV converges to a constant in probability. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | Singular learning machines / Bayes generalization error / Bayes training error / Gibbs generalization error / Gibbs training error |
| Paper # | NC2008-111 |
| Date of Issue |
| Conference Information | |
| Committee | NC |
|---|---|
| Conference Date | 2009/3/4(1days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | |
| Chair | |
| Vice Chair | |
| Secretary | |
| Assistant | |
| Paper Information | |
| Registration To | Neurocomputing (NC) |
|---|---|
| Language | JPN |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | On a Relation between a Limit Theorem in Learning Theory and Singular Fluctuation |
| Sub Title (in English) | |
| Keyword(1) | Singular learning machines |
| Keyword(2) | Bayes generalization error |
| Keyword(3) | Bayes training error |
| Keyword(4) | Gibbs generalization error |
| Keyword(5) | Gibbs training error |
| 1st Author's Name | Sumio WATANABE |
| 1st Author's Affiliation | Tokyo Institute of Technology, PI Lab() |
| Date | 2009-03-11 |
| Paper # | NC2008-111 |
| Volume (vol) | vol.108 |
| Number (no) | 480 |
| Page | pp.pp.- |
| #Pages | 6 |
| Date of Issue | |