| Presentation | 2005/10/11 Analysis of Subspace Bayes Approach in Linear Neural Networks : Relation between Baysian Approach and Shrinkage Estimation Shinichi NAKAJIMA, Sumio WATANABE, |
|---|---|
| PDF Download Page | PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | It is well known that the generalization performance of unidentifiable models differs from that of the regular models. According to recent works, it is known that the Bayes estimation has the advantage over the maximum likelihood estimation. However, accurate approximation of the posterior distribution requires huge computational costs. In this paper, we consider an alternative approximation method, which we call a subspace Bayes approach, and discuss the relation to the shrinkage estimation and the variational Bayes approach. We show that, in three-layer linear neural networks, the subspace Bayes approach is asymptotically equivalent to a positive-part James-Stein type shrinkage estimation, that it provides as good generalization performance as the Bayes estimation in typical cases, and that it is strongly related to the variational Bayes approach. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | subspace Bayes / empirical Bayes / variational Bayes / shrinkage / neural networks / singular model |
| Paper # | NC2005-49 |
| Date of Issue |
| Conference Information | |
| Committee | NC |
|---|---|
| Conference Date | 2005/10/11(1days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | |
| Chair | |
| Vice Chair | |
| Secretary | |
| Assistant | |
| Paper Information | |
| Registration To | Neurocomputing (NC) |
|---|---|
| Language | JPN |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Analysis of Subspace Bayes Approach in Linear Neural Networks : Relation between Baysian Approach and Shrinkage Estimation |
| Sub Title (in English) | |
| Keyword(1) | subspace Bayes |
| Keyword(2) | empirical Bayes |
| Keyword(3) | variational Bayes |
| Keyword(4) | shrinkage |
| Keyword(5) | neural networks |
| Keyword(6) | singular model |
| 1st Author's Name | Shinichi NAKAJIMA |
| 1st Author's Affiliation | Tokyo Institute of Technology:Nikon Corporation() |
| 2nd Author's Name | Sumio WATANABE |
| 2nd Author's Affiliation | Tokyo Institute of Technology |
| Date | 2005/10/11 |
| Paper # | NC2005-49 |
| Volume (vol) | vol.105 |
| Number (no) | 342 |
| Page | pp.pp.- |
| #Pages | 6 |
| Date of Issue | |