Presentation 2006-03-15
Analysis of Hierarchical Variational Bayes Approach in Linear Inverse Problem
Shinichi NAKAJIMA, Sumio WATANABE,
PDF Download Page PDF download Page Link
Abstract(in Japanese) (See Japanese page)
Abstract(in English) It is known that, in singular models, the Bayes estimation commonly has the advantage of generalization performance over the maximum likelihood estimation, however, its accurate approximation requires huge computational costs. The variational Bayes (VB) approach has been proposed as a tractable approximation method of the Bayes estimation, and shown good generalization performance in many applications. Recently, the VB approach has been applied to the automatic relevance determination model (ARD), a kind of hierarchical Bayesian learning, in brain current estimation from MEG data, a practical linear inverse problem. On the other hand, we have been proved that, in three-layer linear neural networks (LNNs), the VB approach is asymptotically equivalent to the James-Stein type shrinkage estimation, and theoretically clarified its generalization performance. In this paper, noting the similarity between the ARD in a linear problem and an LNN, we analyze a simplified version of the VB approach in the ARD. We will show its relation to the shrinkage estimation, and the fact that the relevance determination is caused by a kind of phase transition.
Keyword(in Japanese) (See Japanese page)
Keyword(in English) variational Bayes / hierarchical Bayes / automatic relevance determination / shrinkage estimation / MEG
Paper # NC2005-117
Date of Issue

Conference Information
Committee NC
Conference Date 2006/3/8(1days)
Place (in Japanese) (See Japanese page)
Place (in English)
Topics (in Japanese) (See Japanese page)
Topics (in English)
Chair
Vice Chair
Secretary
Assistant

Paper Information
Registration To Neurocomputing (NC)
Language ENG
Title (in Japanese) (See Japanese page)
Sub Title (in Japanese) (See Japanese page)
Title (in English) Analysis of Hierarchical Variational Bayes Approach in Linear Inverse Problem
Sub Title (in English)
Keyword(1) variational Bayes
Keyword(2) hierarchical Bayes
Keyword(3) automatic relevance determination
Keyword(4) shrinkage estimation
Keyword(5) MEG
1st Author's Name Shinichi NAKAJIMA
1st Author's Affiliation Tokyo Institute of Technology:Nikon Corporation()
2nd Author's Name Sumio WATANABE
2nd Author's Affiliation Tokyo Institute of Technology
Date 2006-03-15
Paper # NC2005-117
Volume (vol) vol.105
Number (no) 657
Page pp.pp.-
#Pages 6
Date of Issue