| Presentation | 2012-11-07 Regularization of Restricted Boltzmann Machine Learning through Entropy Minimization Taichi KIWAKI, Takaki MAKINO, Kazuyuki AIHARA, |
|---|---|
| PDF Download Page |  PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | We propose a learning scheme for Restricted Boltzmann Machines (RBMs) that suppresses over-fitting, where the entropy of the hidden variables is used as a regularization term. Since conventional RBMs lack mechanisms to reduce their redundant complexity, they tend to suffer from over-fitting, that is, learning excessively complex expressions regardless of the complexity of the data. In our approach, the complexity of RBMs is dynamically adjusted to be right for expressing the given data. This can be regarded as an implementation of so-called model selection on RBMs. In this paper, we analytically derived a learning algorithm based on the regularization term using techniques of statistical mechanics. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | Unsupervised learning / Statistical-mechanical informatics / Restricted Boltzmann Machines / Entropy minimization |
| Paper # | IBISML2012-48 |
| Date of Issue |
| Conference Information | |
| Committee | IBISML |
|---|---|
| Conference Date | 2012/10/31(1days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | |
| Chair | |
| Vice Chair | |
| Secretary | |
| Assistant | |
| Paper Information | |
| Registration To | Information-Based Induction Sciences and Machine Learning (IBISML) |
|---|---|
| Language | JPN |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Regularization of Restricted Boltzmann Machine Learning through Entropy Minimization |
| Sub Title (in English) | |
| Keyword(1) | Unsupervised learning |
| Keyword(2) | Statistical-mechanical informatics |
| Keyword(3) | Restricted Boltzmann Machines |
| Keyword(4) | Entropy minimization |
| 1st Author's Name | Taichi KIWAKI |
| 1st Author's Affiliation | Graduated School of Engineering, The University of Tokyo() |
| 2nd Author's Name | Takaki MAKINO |
| 2nd Author's Affiliation | Institute of Industrial Science, The University of Tokyo |
| 3rd Author's Name | Kazuyuki AIHARA |
| 3rd Author's Affiliation | Institute of Industrial Science, The University of Tokyo |
| Date | 2012-11-07 |
| Paper # | IBISML2012-48 |
| Volume (vol) | vol.112 |
| Number (no) | 279 |
| Page | pp.pp.- |
| #Pages | 4 |
| Date of Issue | |