| Presentation | 2022-12-03 Stabilization and Acceleration of Gradient Descent Based on Eigenvalue Decomposition of the Fisher Information Matrix Jun'ichi Takeuchi, Yoshinari Takeishi, Masazumi Iida, Noboru Murata, Kazushi Mimura, Hiroshi Nagaoka, |
|---|---|
| PDF Download Page |  PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | We propose a method to stabilize the gradient decent method without decreasing learning rate for two-layer neural networks, which can accelerate learning process as a result. The method is given by modifying the gradient based on the eigenvalue grouping phenomena for the Fisher information matrix, which was discovered by the authors. We report that our method is effective by numerical simulation. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | fisher information matrix / eigenvalu decomposition / gradient decent / learning rate |
| Paper # | MBE2022-39,NC2022-61 |
| Date of Issue | 2022-11-26 (MBE, NC) |
| Conference Information | |
| Committee | MBE / NC |
|---|---|
| Conference Date | 2022/12/3(1days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | Osaka Electro-Communication University |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | NC, ME, etc. |
| Chair | Junichi Hori(Niigata Univ.) / Hiroshi Yamakawa(Univ of Tokyo) |
| Vice Chair | Hisashi Yoshida(Kinki Univ.) / Hirokazu Tanaka(Tokyo City Univ.) |
| Secretary | Hisashi Yoshida(Setsunan Univ) / Hirokazu Tanaka(Tohoku Inst. of Tech.) |
| Assistant | Emi Yuda(Tohoku Univ) / Miki Kaneko(Osaka Univ.) / Yoshimasa Tawatsuji(Waseda Univ.) / Tomoki Kurikawa(KMU) |
| Paper Information | |
| Registration To | Technical Committee on ME and Bio Cybernetics / Technical Committee on Neurocomputing |
|---|---|
| Language | JPN |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Stabilization and Acceleration of Gradient Descent Based on Eigenvalue Decomposition of the Fisher Information Matrix |
| Sub Title (in English) | |
| Keyword(1) | fisher information matrix |
| Keyword(2) | eigenvalu decomposition |
| Keyword(3) | gradient decent |
| Keyword(4) | learning rate |
| 1st Author's Name | Jun'ichi Takeuchi |
| 1st Author's Affiliation | Kyushu University(Kyushu Univ.) |
| 2nd Author's Name | Yoshinari Takeishi |
| 2nd Author's Affiliation | Kyushu University(Kyushu Univ.) |
| 3rd Author's Name | Masazumi Iida |
| 3rd Author's Affiliation | Kyushu University(Kyushu Univ.) |
| 4th Author's Name | Noboru Murata |
| 4th Author's Affiliation | Waseda University(Waseda Univ.) |
| 5th Author's Name | Kazushi Mimura |
| 5th Author's Affiliation | Hiroshima City University(Hiroshima City Univ.) |
| 6th Author's Name | Hiroshi Nagaoka |
| 6th Author's Affiliation | University of Electro Communication(Univ. of Electro Communication) |
| Date | 2022-12-03 |
| Paper # | MBE2022-39,NC2022-61 |
| Volume (vol) | vol.122 |
| Number (no) | MBE-291,NC-292 |
| Page | pp.pp.80-85(MBE), pp.80-85(NC), |
| #Pages | 6 |
| Date of Issue | 2022-11-26 (MBE, NC) |