Paper Abstract and Keywords |
Presentation |
2017-01-27 13:25
A study on multilayered neural network with simultaneous perturbation learning rule Kenji Onoue, Hidetaka Ito, Hiroomi Hikawa (Kansai Univ) NC2016-58 |
Abstract |
(in Japanese) |
(See Japanese page) |
(in English) |
This paper describes a study on multilayered neural network with simultaneous perturbation learning rule.
The learning rule used here is kind of a gradient descent method, and it's modifying quantities are stochastically obtained.
The learning rule requires only forward operations of the neural network.
In comparison with back propagation algorithm, simultaneous perturbation learning rule does not need backward operation.
Therefore, a very simple system can implement the training.
Also, simultaneous perturbation learning rule is expected to solve gradient vanishing problem because size of modifying quantities in each layer are similar.
However simultaneous learning rule has the problem that modifying quantity is bigger than the expected value.
To solve that problem, we used the average of some modifying quantities calculated with different perturbations.
We implemented the simultaneous perturbation learning rule that is robust to the gradient vanishing problem. |
Keyword |
(in Japanese) |
(See Japanese page) |
(in English) |
multilayered neuralnetwork / simultaneous perturbation learning rule / gradient vanishing problem / MNIST / / / / |
Reference Info. |
IEICE Tech. Rep., vol. 116, no. 424, NC2016-58, pp. 59-64, Jan. 2017. |
Paper # |
NC2016-58 |
Date of Issue |
2017-01-19 (NC) |
ISSN |
Print edition: ISSN 0913-5685 Online edition: ISSN 2432-6380 |
Copyright and reproduction |
All rights are reserved and no part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy, recording, or any information storage and retrieval system, without permission in writing from the publisher. Notwithstanding, instructors are permitted to photocopy isolated articles for noncommercial classroom use without fee. (License No.: 10GA0019/12GB0052/13GB0056/17GB0034/18GB0034) |
Download PDF |
NC2016-58 |
Conference Information |
Committee |
NC NLP |
Conference Date |
2017-01-26 - 2017-01-27 |
Place (in Japanese) |
(See Japanese page) |
Place (in English) |
Kitakyushu Foundation for the Advanement of Ind. Sci. and Tech. |
Topics (in Japanese) |
(See Japanese page) |
Topics (in English) |
Implementation of Neuro Computing,Analysis and Modeling of Human Science, etc |
Paper Information |
Registration To |
NC |
Conference Code |
2017-01-NC-NLP |
Language |
Japanese |
Title (in Japanese) |
(See Japanese page) |
Sub Title (in Japanese) |
(See Japanese page) |
Title (in English) |
A study on multilayered neural network with simultaneous perturbation learning rule |
Sub Title (in English) |
|
Keyword(1) |
multilayered neuralnetwork |
Keyword(2) |
simultaneous perturbation learning rule |
Keyword(3) |
gradient vanishing problem |
Keyword(4) |
MNIST |
Keyword(5) |
|
Keyword(6) |
|
Keyword(7) |
|
Keyword(8) |
|
1st Author's Name |
Kenji Onoue |
1st Author's Affiliation |
Kansai University (Kansai Univ) |
2nd Author's Name |
Hidetaka Ito |
2nd Author's Affiliation |
Kansai University (Kansai Univ) |
3rd Author's Name |
Hiroomi Hikawa |
3rd Author's Affiliation |
Kansai University (Kansai Univ) |
4th Author's Name |
|
4th Author's Affiliation |
() |
5th Author's Name |
|
5th Author's Affiliation |
() |
6th Author's Name |
|
6th Author's Affiliation |
() |
7th Author's Name |
|
7th Author's Affiliation |
() |
8th Author's Name |
|
8th Author's Affiliation |
() |
9th Author's Name |
|
9th Author's Affiliation |
() |
10th Author's Name |
|
10th Author's Affiliation |
() |
11th Author's Name |
|
11th Author's Affiliation |
() |
12th Author's Name |
|
12th Author's Affiliation |
() |
13th Author's Name |
|
13th Author's Affiliation |
() |
14th Author's Name |
|
14th Author's Affiliation |
() |
15th Author's Name |
|
15th Author's Affiliation |
() |
16th Author's Name |
|
16th Author's Affiliation |
() |
17th Author's Name |
|
17th Author's Affiliation |
() |
18th Author's Name |
|
18th Author's Affiliation |
() |
19th Author's Name |
|
19th Author's Affiliation |
() |
20th Author's Name |
|
20th Author's Affiliation |
() |
Speaker |
Author-1 |
Date Time |
2017-01-27 13:25:00 |
Presentation Time |
25 minutes |
Registration for |
NC |
Paper # |
NC2016-58 |
Volume (vol) |
vol.116 |
Number (no) |
no.424 |
Page |
pp.59-64 |
#Pages |
6 |
Date of Issue |
2017-01-19 (NC) |
|