| Presentation | 2006-12-05 On Variational Message Passing and Its Relation to Other Message Passing Inference Algorithms Justin Dauwels, |
|---|---|
| PDF Download Page |  PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | Variational methods are frequently used for performing inference in graphical models. The sum-product algorithm is often intractable for systems with continuous variables, and variational methods are then an interesting alternative; moreover, variational methods are guaranteed to convergence. On the other hand, it is well known that for discrete variables, the sum-product algorithm usually leads to better results than variational methods (since the Bethe approximation to the Gibbs free energy is more accurate than the mean field approximation); however, the sum-product algorithm is not guaranteed to convergence on cyclic graphs. Those considerations naturally lead to the following questions: Can one perhaps mix both approaches in one algorithm in order to combine their strengths? Which of those combinations lead to variational methods, and hence, are guaranteed to converge? Can such combinations be derived directly from a factor graph of the system at hand by mechanically applying message computation rules, in this way. bypassing error-prone variational calculus? In this note, an answer to those question is outlined. The key idea is to formulate variational methods as message passing algorithms operating on factor graphs, elaborating on earlier work by Beal et al., Winn et al. and Xing et al. In this note, it is also shown that the variational message passing algorithm is strongly related to the message passing formulation of expectation maximization (EM). It is demonstrated how variational methods can be combined with various other message passing algorithms, e.g., Kalman filters and smoothers, iterated conditional modes, EM, gradient methods, and particle filters. Some of those combinations have been explored in the literature, others seem to be new. Generic message computation rules for such combinations are formulated. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | |
| Paper # | NC2006-84 |
| Date of Issue |
| Conference Information | |
| Committee | NC |
|---|---|
| Conference Date | 2006/11/28(1days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | |
| Chair | |
| Vice Chair | |
| Secretary | |
| Assistant | |
| Paper Information | |
| Registration To | Neurocomputing (NC) |
|---|---|
| Language | ENG |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | On Variational Message Passing and Its Relation to Other Message Passing Inference Algorithms |
| Sub Title (in English) | |
| Keyword(1) | |
| 1st Author's Name | Justin Dauwels |
| 1st Author's Affiliation | RIKEN Brain Science Institute() |
| Date | 2006-12-05 |
| Paper # | NC2006-84 |
| Volume (vol) | vol.106 |
| Number (no) | 407 |
| Page | pp.pp.- |
| #Pages | 6 |
| Date of Issue | |