| Presentation | 2008-06-27 Application of Adaptive Natural Gradient Descent to Singular Learning Machines Yoshiya NISHIMURA, Masato INOUE, |
|---|---|
| PDF Download Page |  PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | Fisher information matrix degenerates at singular points in singular learning machines (SLM). Therefore, the adaptive natural gradient descent (ANGD) breaks down because the estimated inverse Fisher information matrix is diverged. In this research, we propose a general method avoiding the divergence of the Fisher information matrix by focusing on one aspect of this problem, i.e., the divergence is caused by the limit of the numerical precision. We have validated the proposed method using soft committee machines. Compared to the original ANGD, the proposed method successfully avoided the divergence without any significant degradation of both the training speed and the calculation cost. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | adaptive natural gradient descent / singular learning machine / soft committee machine |
| Paper # | NC2008-25 |
| Date of Issue |
| Conference Information | |
| Committee | NC |
|---|---|
| Conference Date | 2008/6/19(1days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | |
| Chair | |
| Vice Chair | |
| Secretary | |
| Assistant | |
| Paper Information | |
| Registration To | Neurocomputing (NC) |
|---|---|
| Language | JPN |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Application of Adaptive Natural Gradient Descent to Singular Learning Machines |
| Sub Title (in English) | |
| Keyword(1) | adaptive natural gradient descent |
| Keyword(2) | singular learning machine |
| Keyword(3) | soft committee machine |
| 1st Author's Name | Yoshiya NISHIMURA |
| 1st Author's Affiliation | Dept. of Electrical Engineering and Bioscience, Graduate School of Advanced Science and Engineering, Waseda University() |
| 2nd Author's Name | Masato INOUE |
| 2nd Author's Affiliation | Dept. of Electrical Engineering and Bioscience, Graduate School of Advanced Science and Engineering, Waseda University |
| Date | 2008-06-27 |
| Paper # | NC2008-25 |
| Volume (vol) | vol.108 |
| Number (no) | 101 |
| Page | pp.pp.- |
| #Pages | 5 |
| Date of Issue | |