Presentation 2017-01-27
A study on multilayered neural network with simultaneous perturbation learning rule
Kenji Onoue, Hidetaka Ito, Hiroomi Hikawa,
PDF Download Page PDF download Page Link
Abstract(in Japanese) (See Japanese page)
Abstract(in English) This paper describes a study on multilayered neural network with simultaneous perturbation learning rule. The learning rule used here is kind of a gradient descent method, and it's modifying quantities are stochastically obtained. The learning rule requires only forward operations of the neural network. In comparison with back propagation algorithm, simultaneous perturbation learning rule does not need backward operation. Therefore, a very simple system can implement the training. Also, simultaneous perturbation learning rule is expected to solve gradient vanishing problem because size of modifying quantities in each layer are similar. However simultaneous learning rule has the problem that modifying quantity is bigger than the expected value. To solve that problem, we used the average of some modifying quantities calculated with different perturbations. We implemented the simultaneous perturbation learning rule that is robust to the gradient vanishing problem.
Keyword(in Japanese) (See Japanese page)
Keyword(in English) multilayered neuralnetwork / simultaneous perturbation learning rule / gradient vanishing problem / MNIST
Paper # NC2016-58
Date of Issue 2017-01-19 (NC)

Conference Information
Committee NC / NLP
Conference Date 2017/1/26(2days)
Place (in Japanese) (See Japanese page)
Place (in English) Kitakyushu Foundation for the Advanement of Ind. Sci. and Tech.
Topics (in Japanese) (See Japanese page)
Topics (in English) Implementation of Neuro Computing,Analysis and Modeling of Human Science, etc
Chair Shigeo Sato(Tohoku Univ.) / Hisato Fujisaka(Hiroshima City Univ.)
Vice Chair Masafumi Hagiwara(Keio Univ.) / Masaharu Adachi(Tokyo Denki Univ.)
Secretary Masafumi Hagiwara(Kyoto Sangyo Univ.) / Masaharu Adachi(Tokyo Inst. of Tech.)
Assistant Hisanao Akima(Tohoku Univ.) / Yoshihisa Shinozawa(Keio Univ.) / Hiroyuki Asahara(Okayama Univ. of Science) / Toshihiro Tachibana(Shonan Inst. of Tech.)

Paper Information
Registration To Technical Committee on Neurocomputing / Technical Committee on Nonlinear Problems
Language JPN
Title (in Japanese) (See Japanese page)
Sub Title (in Japanese) (See Japanese page)
Title (in English) A study on multilayered neural network with simultaneous perturbation learning rule
Sub Title (in English)
Keyword(1) multilayered neuralnetwork
Keyword(2) simultaneous perturbation learning rule
Keyword(3) gradient vanishing problem
Keyword(4) MNIST
1st Author's Name Kenji Onoue
1st Author's Affiliation Kansai University(Kansai Univ)
2nd Author's Name Hidetaka Ito
2nd Author's Affiliation Kansai University(Kansai Univ)
3rd Author's Name Hiroomi Hikawa
3rd Author's Affiliation Kansai University(Kansai Univ)
Date 2017-01-27
Paper # NC2016-58
Volume (vol) vol.116
Number (no) NC-424
Page pp.pp.59-64(NC),
#Pages 6
Date of Issue 2017-01-19 (NC)