| Presentation | 2014-11-18 Asymptotic Analysis of Variational Bayesian Latent Dirichlet Allocation Shinichi NAKAJIMA, Issei SATO, Masashi SUGIYAMA, Kazuho WATANABE, Hiroko KOBAYASHI, |
|---|---|
| PDF Download Page |  PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | Latent Dirichlet allocation (LDA) is a popular generative model of various objects such as texts and images, where an object is expressed as a mixture of latent topics. In this paper, we theoretically investigate variational Bayesian (VB) learning in LDA. More specifically, we analytically derive the leading term of the VB free energy under an asymptotic setup, and show that there exist transition thresholds in Dirichlet hyperparameters around which the sparsity-inducing behavior drastically changes. Then we further theoretically reveal the notable phenomenon that VB tends to induce weaker sparsity than MAP in the LDA model, which is opposed to other models. We experimentally demonstrate the practical validity of our asymptotic theory on real-world Last.FM music data. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | latent Dirichlet allocation / variational Bayes / free energy / partially Bayes / sparsity |
| Paper # | IBISML2014-64 |
| Date of Issue |
| Conference Information | |
| Committee | IBISML |
|---|---|
| Conference Date | 2014/11/10(1days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | |
| Chair | |
| Vice Chair | |
| Secretary | |
| Assistant | |
| Paper Information | |
| Registration To | Information-Based Induction Sciences and Machine Learning (IBISML) |
|---|---|
| Language | ENG |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Asymptotic Analysis of Variational Bayesian Latent Dirichlet Allocation |
| Sub Title (in English) | |
| Keyword(1) | latent Dirichlet allocation |
| Keyword(2) | variational Bayes |
| Keyword(3) | free energy |
| Keyword(4) | partially Bayes |
| Keyword(5) | sparsity |
| 1st Author's Name | Shinichi NAKAJIMA |
| 1st Author's Affiliation | Berlin Big Data Center, Technische Universitat Berlin() |
| 2nd Author's Name | Issei SATO |
| 2nd Author's Affiliation | University of Tokyo |
| 3rd Author's Name | Masashi SUGIYAMA |
| 3rd Author's Affiliation | University of Tokyo |
| 4th Author's Name | Kazuho WATANABE |
| 4th Author's Affiliation | Toyohashi University of Technology |
| 5th Author's Name | Hiroko KOBAYASHI |
| 5th Author's Affiliation | Nikon Corporation |
| Date | 2014-11-18 |
| Paper # | IBISML2014-64 |
| Volume (vol) | vol.114 |
| Number (no) | 306 |
| Page | pp.pp.- |
| #Pages | 8 |
| Date of Issue | |