| Presentation | 2006-03-15 Analysis of Hierarchical Variational Bayes Approach in Linear Inverse Problem Shinichi NAKAJIMA, Sumio WATANABE, |
|---|---|
| PDF Download Page |  PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | It is known that, in singular models, the Bayes estimation commonly has the advantage of generalization performance over the maximum likelihood estimation, however, its accurate approximation requires huge computational costs. The variational Bayes (VB) approach has been proposed as a tractable approximation method of the Bayes estimation, and shown good generalization performance in many applications. Recently, the VB approach has been applied to the automatic relevance determination model (ARD), a kind of hierarchical Bayesian learning, in brain current estimation from MEG data, a practical linear inverse problem. On the other hand, we have been proved that, in three-layer linear neural networks (LNNs), the VB approach is asymptotically equivalent to the James-Stein type shrinkage estimation, and theoretically clarified its generalization performance. In this paper, noting the similarity between the ARD in a linear problem and an LNN, we analyze a simplified version of the VB approach in the ARD. We will show its relation to the shrinkage estimation, and the fact that the relevance determination is caused by a kind of phase transition. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | variational Bayes / hierarchical Bayes / automatic relevance determination / shrinkage estimation / MEG |
| Paper # | NC2005-117 |
| Date of Issue |
| Conference Information | |
| Committee | NC |
|---|---|
| Conference Date | 2006/3/8(1days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | |
| Chair | |
| Vice Chair | |
| Secretary | |
| Assistant | |
| Paper Information | |
| Registration To | Neurocomputing (NC) |
|---|---|
| Language | ENG |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Analysis of Hierarchical Variational Bayes Approach in Linear Inverse Problem |
| Sub Title (in English) | |
| Keyword(1) | variational Bayes |
| Keyword(2) | hierarchical Bayes |
| Keyword(3) | automatic relevance determination |
| Keyword(4) | shrinkage estimation |
| Keyword(5) | MEG |
| 1st Author's Name | Shinichi NAKAJIMA |
| 1st Author's Affiliation | Tokyo Institute of Technology:Nikon Corporation() |
| 2nd Author's Name | Sumio WATANABE |
| 2nd Author's Affiliation | Tokyo Institute of Technology |
| Date | 2006-03-15 |
| Paper # | NC2005-117 |
| Volume (vol) | vol.105 |
| Number (no) | 657 |
| Page | pp.pp.- |
| #Pages | 6 |
| Date of Issue | |