| Presentation | 2014-11-21 The Relation between Dispersion of Initial Values and Pre-training of Deep Neural Networks Sitaro SHINAGAWA, Yoshihiro HAYAKAWA, Takeshi ONOMI, Koji NAKAJIMA, |
|---|---|
| PDF Download Page |  PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | Greedy Layer-wise Pre-training (Pre-training) is known as a major method to construct Deep Neural Networks (DNN) with Deep Learning. Restricted Boltzmann Machine (RBM) or Auto-Encoder (AE) is frequently used for the Pre-training. After Pre-training,we can use the weights and the biases learned through Pre-training as initial values for DNN with high performance. The reason why Pre-training makes good initial values of DNN has been discussed at various viewpoints, but there is no ideas to the aspect of the relation between dispersion of initial values.In this study, we consider the dispersion of parameters generated through Pre-training. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | Newral network / Hierarchical / Deep learning / Pre-training / Restricted Boltzmann Machine / Dispersion |
| Paper # | NC2014-27 |
| Date of Issue |
| Conference Information | |
| Committee | NC |
|---|---|
| Conference Date | 2014/11/14(1days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | |
| Chair | |
| Vice Chair | |
| Secretary | |
| Assistant | |
| Paper Information | |
| Registration To | Neurocomputing (NC) |
|---|---|
| Language | JPN |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | The Relation between Dispersion of Initial Values and Pre-training of Deep Neural Networks |
| Sub Title (in English) | |
| Keyword(1) | Newral network |
| Keyword(2) | Hierarchical |
| Keyword(3) | Deep learning |
| Keyword(4) | Pre-training |
| Keyword(5) | Restricted Boltzmann Machine |
| Keyword(6) | Dispersion |
| 1st Author's Name | Sitaro SHINAGAWA |
| 1st Author's Affiliation | Laboratory for Brainware Research Institute of Electrical Communication, Tohoku University:Laboratory for Nanoelectoronics and Spintoronics Reserch Institute of Electorical Communication, Tohoku University First University() |
| 2nd Author's Name | Yoshihiro HAYAKAWA |
| 2nd Author's Affiliation | Sendai National College of Technology |
| 3rd Author's Name | Takeshi ONOMI |
| 3rd Author's Affiliation | Laboratory for Brainware Research Institute of Electrical Communication, Tohoku University:Laboratory for Nanoelectoronics and Spintoronics Reserch Institute of Electorical Communication, Tohoku University First University |
| 4th Author's Name | Koji NAKAJIMA |
| 4th Author's Affiliation | Laboratory for Brainware Research Institute of Electrical Communication, Tohoku University:Laboratory for Nanoelectoronics and Spintoronics Reserch Institute of Electorical Communication, Tohoku University First University |
| Date | 2014-11-21 |
| Paper # | NC2014-27 |
| Volume (vol) | vol.114 |
| Number (no) | 326 |
| Page | pp.pp.- |
| #Pages | 4 |
| Date of Issue | |