| Presentation | 2022-03-02 Learning of a stacked autoencoder with regularizers added to the cost function, evaluation of their effectiveness, and clarification of its information compression mechanism Masumi Ishikawa, |
|---|---|
| PDF Download Page |  PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | Deep learning has a serious drawback in that the resulting models tend to be a black box, hence hard to understand. A sparse modeling approach is expected to ameliorate the drawback. Various regularization terms are proposed so far. The paper proposes to use the concept of Pareto optimality composed of data fitting and the sparseness of models for judging the effectiveness of regularization terms using DB such as US census data. We have demonstrated that compared to the most popular L1-norm to connection weights, the selective L1-norm to connection weights is more effective, the selective L1 norm or L2 norm to hidden outputs and KL divergence or off-diagonal squared covariance of hidden outputs are yet more effective. This enables the clarification of information compression mechanism of the resulting stacked autoencoders. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | Deep learning / Black-box model / Stacked autoencoder / Explainable / Sparse modeling / Regularizers |
| Paper # | NC2021-49 |
| Date of Issue | 2022-02-23 (NC) |
| Conference Information | |
| Committee | MBE / NC |
|---|---|
| Conference Date | 2022/3/2(3days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | Online |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | |
| Chair | Ryuhei Okuno(Setsunan Univ.) / Rieko Osu(Waseda Univ.) |
| Vice Chair | Junichi Hori(Niigata Univ.) / Hiroshi Yamakawa(Univ of Tokyo) |
| Secretary | Junichi Hori(Osaka Electro-Communication Univ) / Hiroshi Yamakawa(ATR) |
| Assistant | Jun Akazawa(Meiji Univ. of Integrative Medicine) / Emi Yuda(Tohoku Univ) / Nobuhiko Wagatsuma(Toho Univ.) / Tomoki Kurikawa(KMU) |
| Paper Information | |
| Registration To | Technical Committee on ME and Bio Cybernetics / Technical Committee on Neurocomputing |
|---|---|
| Language | JPN |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Learning of a stacked autoencoder with regularizers added to the cost function, evaluation of their effectiveness, and clarification of its information compression mechanism |
| Sub Title (in English) | |
| Keyword(1) | Deep learning |
| Keyword(2) | Black-box model |
| Keyword(3) | Stacked autoencoder |
| Keyword(4) | Explainable |
| Keyword(5) | Sparse modeling |
| Keyword(6) | Regularizers |
| 1st Author's Name | Masumi Ishikawa |
| 1st Author's Affiliation | Kyushu Institute of Technology(Kyutech) |
| Date | 2022-03-02 |
| Paper # | NC2021-49 |
| Volume (vol) | vol.121 |
| Number (no) | NC-390 |
| Page | pp.pp.17-22(NC), |
| #Pages | 6 |
| Date of Issue | 2022-02-23 (NC) |