Presentation | 2017-01-27 A study on multilayered neural network with simultaneous perturbation learning rule Kenji Onoue, Hidetaka Ito, Hiroomi Hikawa, |
---|---|
PDF Download Page | PDF download Page Link |
Abstract(in Japanese) | (See Japanese page) |
Abstract(in English) | This paper describes a study on multilayered neural network with simultaneous perturbation learning rule. The learning rule used here is kind of a gradient descent method, and it's modifying quantities are stochastically obtained. The learning rule requires only forward operations of the neural network. In comparison with back propagation algorithm, simultaneous perturbation learning rule does not need backward operation. Therefore, a very simple system can implement the training. Also, simultaneous perturbation learning rule is expected to solve gradient vanishing problem because size of modifying quantities in each layer are similar. However simultaneous learning rule has the problem that modifying quantity is bigger than the expected value. To solve that problem, we used the average of some modifying quantities calculated with different perturbations. We implemented the simultaneous perturbation learning rule that is robust to the gradient vanishing problem. |
Keyword(in Japanese) | (See Japanese page) |
Keyword(in English) | multilayered neuralnetwork / simultaneous perturbation learning rule / gradient vanishing problem / MNIST |
Paper # | NC2016-58 |
Date of Issue | 2017-01-19 (NC) |
Conference Information | |
Committee | NC / NLP |
---|---|
Conference Date | 2017/1/26(2days) |
Place (in Japanese) | (See Japanese page) |
Place (in English) | Kitakyushu Foundation for the Advanement of Ind. Sci. and Tech. |
Topics (in Japanese) | (See Japanese page) |
Topics (in English) | Implementation of Neuro Computing,Analysis and Modeling of Human Science, etc |
Chair | Shigeo Sato(Tohoku Univ.) / Hisato Fujisaka(Hiroshima City Univ.) |
Vice Chair | Masafumi Hagiwara(Keio Univ.) / Masaharu Adachi(Tokyo Denki Univ.) |
Secretary | Masafumi Hagiwara(Kyoto Sangyo Univ.) / Masaharu Adachi(Tokyo Inst. of Tech.) |
Assistant | Hisanao Akima(Tohoku Univ.) / Yoshihisa Shinozawa(Keio Univ.) / Hiroyuki Asahara(Okayama Univ. of Science) / Toshihiro Tachibana(Shonan Inst. of Tech.) |
Paper Information | |
Registration To | Technical Committee on Neurocomputing / Technical Committee on Nonlinear Problems |
---|---|
Language | JPN |
Title (in Japanese) | (See Japanese page) |
Sub Title (in Japanese) | (See Japanese page) |
Title (in English) | A study on multilayered neural network with simultaneous perturbation learning rule |
Sub Title (in English) | |
Keyword(1) | multilayered neuralnetwork |
Keyword(2) | simultaneous perturbation learning rule |
Keyword(3) | gradient vanishing problem |
Keyword(4) | MNIST |
1st Author's Name | Kenji Onoue |
1st Author's Affiliation | Kansai University(Kansai Univ) |
2nd Author's Name | Hidetaka Ito |
2nd Author's Affiliation | Kansai University(Kansai Univ) |
3rd Author's Name | Hiroomi Hikawa |
3rd Author's Affiliation | Kansai University(Kansai Univ) |
Date | 2017-01-27 |
Paper # | NC2016-58 |
Volume (vol) | vol.116 |
Number (no) | NC-424 |
Page | pp.pp.59-64(NC), |
#Pages | 6 |
Date of Issue | 2017-01-19 (NC) |