| Presentation | 2023-06-30 Analysis of Mode Connectivity Between Models with Different Hidden Layer Widths Yusuke Takase, Hidetoshi Shimodaira, |
|---|---|
| PDF Download Page |  PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | The property known as mode connectivity, where the weights of two neural networks independently trained using the same dataset and model can be connected with curves in parameter space while keeping the loss small, has been experimentally confirmed. Previous studies have shown linear mode connectivity, which connects weights linearly, by eliminating the permutation symmetry of hidden layers. These studies, however, was premised on identical model structures and parameter sizes. This study extends existing methods and demonstrates that linear mode connectivity can be achieved even when the number of units in the hidden layers of the two models differs. This is shown in experiments using multilayer perceptrons trained on the MNIST dataset and ResNet20 trained on the CIFAR100 dataset. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | neural networks / weight matching / mode connectivity / permutation symmetry / loss landscape visualization / unbalanced assignment |
| Paper # | NC2023-20,IBISML2023-20 |
| Date of Issue | 2023-06-22 (NC, IBISML) |
| Conference Information | |
| Committee | NC / IBISML / IPSJ-BIO / IPSJ-MPS |
|---|---|
| Conference Date | 2023/6/29(3days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | OIST Conference Center |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | |
| Chair | Hirokazu Tanaka(Tokyo City Univ.) / Masashi Sugiyama(Univ. of Tokyo) |
| Vice Chair | Jun Izawa(Univ. of Tsukub) / Toshihiro Kamishima(AIST) / Koji Tsuda(Univ. of Tokyo) |
| Secretary | Jun Izawa(NTT) / Toshihiro Kamishima(NAIST) / Koji Tsuda(NTT) / (Hokkaido Univ.) |
| Assistant | Yoshimasa Tawatsuji(Waseda Univ.) / Takato Horii(Osaka Univ.) / Yoshinobu Kawahara(Osaka Univ.) / Taiji Suzuki(Tokyo Inst. of Tech.) |
| Paper Information | |
| Registration To | Technical Committee on Neurocomputing / Technical Committee on Information-Based Induction Sciences and Machine Learning / Special Interest Group on Bioinformatics and Genomics / Special Interest Group on Mathematical Modeling and Problem Solving |
|---|---|
| Language | JPN |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Analysis of Mode Connectivity Between Models with Different Hidden Layer Widths |
| Sub Title (in English) | |
| Keyword(1) | neural networks |
| Keyword(2) | weight matching |
| Keyword(3) | mode connectivity |
| Keyword(4) | permutation symmetry |
| Keyword(5) | loss landscape visualization |
| Keyword(6) | unbalanced assignment |
| 1st Author's Name | Yusuke Takase |
| 1st Author's Affiliation | Kyoto University(Kyoto Univ.) |
| 2nd Author's Name | Hidetoshi Shimodaira |
| 2nd Author's Affiliation | Kyoto University(Kyoto Univ.) |
| Date | 2023-06-30 |
| Paper # | NC2023-20,IBISML2023-20 |
| Volume (vol) | vol.123 |
| Number (no) | NC-90,IBISML-91 |
| Page | pp.pp.129-136(NC), pp.129-136(IBISML), |
| #Pages | 8 |
| Date of Issue | 2023-06-22 (NC, IBISML) |