| Presentation | 2021-10-28 A Deep Neural Network Model Compression with Spherical Clustering of Neurons Shin Sakamoto, Masao Okita, Fumihiko Ino, |
|---|---|
| PDF Download Page |  PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | In this paper, we propose weight matrix compression with spherical clustering of neurons , aiming at reducing memory usage and computational complexity while maintaining the accuracy of deep neural network (DNN) inference models. Compared with existing methods that use general k-means clustering, the proposed method reduces the information loss with compression by unit spherizing the weight vector space in advance. Furthermore, the proposed method embeds the vector norm separated by spherization into the weight matrix of the previous layer, in order to obtain the identical calculation results without additional computation and space. We conducted experiments with AlexNet of the DNN model to compare the predictability of compressed DNN models with the proposed method and an existing method. Experimental results show that the proposed method can improve the accuracy of prediction by up to 20 at the same column reduction ratio in the case of high sparsity of the weight matrix before compression. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | information losspruning / k-means / fully connected layer |
| Paper # | NC2021-22 |
| Date of Issue | 2021-10-21 (NC) |
| Conference Information | |
| Committee | MBE / NC |
|---|---|
| Conference Date | 2021/10/28(2days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | Online |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | |
| Chair | Ryuhei Okuno(Setsunan Univ.) / Rieko Osu(Waseda Univ.) |
| Vice Chair | Junichi Hori(Niigata Univ.) / Hiroshi Yamakawa(Univ of Tokyo) |
| Secretary | Junichi Hori(Osaka Electro-Communication Univ) / Hiroshi Yamakawa(ATR) |
| Assistant | Jun Akazawa(Meiji Univ. of Integrative Medicine) / Emi Yuda(Tohoku Univ) / Nobuhiko Wagatsuma(Toho Univ.) / Tomoki Kurikawa(KMU) |
| Paper Information | |
| Registration To | Technical Committee on ME and Bio Cybernetics / Technical Committee on Neurocomputing |
|---|---|
| Language | JPN |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | A Deep Neural Network Model Compression with Spherical Clustering of Neurons |
| Sub Title (in English) | |
| Keyword(1) | information losspruning |
| Keyword(2) | k-means |
| Keyword(3) | fully connected layer |
| 1st Author's Name | Shin Sakamoto |
| 1st Author's Affiliation | Osaka University(Osaka Univ) |
| 2nd Author's Name | Masao Okita |
| 2nd Author's Affiliation | Osaka University(Osaka Univ) |
| 3rd Author's Name | Fumihiko Ino |
| 3rd Author's Affiliation | Osaka University(Osaka Univ) |
| Date | 2021-10-28 |
| Paper # | NC2021-22 |
| Volume (vol) | vol.121 |
| Number (no) | NC-223 |
| Page | pp.pp.22-27(NC), |
| #Pages | 6 |
| Date of Issue | 2021-10-21 (NC) |