| Presentation | 2023-03-15 Proposal of Node Fusion in Sparse DenseNet Shogo Taneda, Shoma Noguchi, Yukari Yamauchi, |
|---|---|
| PDF Download Page |  PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | Gao Huang et al. proposed a deep learning model called DenseNet. This deep learning model successfully prevents information loss by preserving and maintaining information by concatenating the front and back layers. In this study, we introduce L1 regularization in the convolutional part of DenseNet to sparsify the learning parameters and improve the generalization performance. We also introduce L1 regularization for fully connected layers to sparsify the learning parameters. In addition, we propose Node Fusion to sparsify units in fully connected layers. First, the node in the hidden layer with the smallest sum of squares of weights connected to the previous and next layers is targeted for deletion. Delete the node after fusing the weight to a node with high similarity to the node to be deleted. We aim to reduce computation and improve accuracy by Node Fusion. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | DenseNet / Sparse / L1-Regularization / Node Fusion |
| Paper # | NC2022-110 |
| Date of Issue | 2023-03-06 (NC) |
| Conference Information | |
| Committee | NC / MBE |
|---|---|
| Conference Date | 2023/3/13(3days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | The Univ. of Electro-Communications |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | Brain architecture, General |
| Chair | Hiroshi Yamakawa(Univ of Tokyo) / Junichi Hori(Niigata Univ.) |
| Vice Chair | Hirokazu Tanaka(Tokyo City Univ.) / Hisashi Yoshida(Kinki Univ.) |
| Secretary | Hirokazu Tanaka(NTT) / Hisashi Yoshida(NICT) |
| Assistant | Yoshimasa Tawatsuji(Waseda Univ.) / Tomoki Kurikawa(KMU) / Emi Yuda(Tohoku Univ) / Miki Kaneko(Osaka Univ.) |
| Paper Information | |
| Registration To | Technical Committee on Neurocomputing / Technical Committee on ME and Bio Cybernetics |
|---|---|
| Language | JPN |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Proposal of Node Fusion in Sparse DenseNet |
| Sub Title (in English) | |
| Keyword(1) | DenseNet |
| Keyword(2) | Sparse |
| Keyword(3) | L1-Regularization |
| Keyword(4) | Node Fusion |
| 1st Author's Name | Shogo Taneda |
| 1st Author's Affiliation | Nihon University(Nihon Univ.) |
| 2nd Author's Name | Shoma Noguchi |
| 2nd Author's Affiliation | Nihon University(Nihon Univ.) |
| 3rd Author's Name | Yukari Yamauchi |
| 3rd Author's Affiliation | Nihon University(Nihon Univ.) |
| Date | 2023-03-15 |
| Paper # | NC2022-110 |
| Volume (vol) | vol.122 |
| Number (no) | NC-425 |
| Page | pp.pp.105-108(NC), |
| #Pages | 4 |
| Date of Issue | 2023-03-06 (NC) |