| Presentation | 2022-01-23 Adversarial Training with Knowledge Distillation considering Intermediate Feature Representation in CNNs Hikaru Higuchi, Satoshi Suzuki, Hayaru Shouno, |
|---|---|
| PDF Download Page |  PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | Adversarial examples are one of the vulnerability attacks to the convolution neural network (CNN). The adversarialexamples are made by adding adversarial perturbations, which are maliciously designed to deceive the target DNN and aregenerally human-imperceptible, to input images. Adversarial training is a method to improve classification accuracy againstadversarial attacks. In the adversarial training, the CNN is trained with not clean images (not including adversarial pertur-bations) but adversarial examples. However, conventional adversarial training decreases the classification accuracy on cleanimages than usual training which trains the CNN with clean images only. From our experimental results, the CNNs trained onclean images only can obtain effective feature representations for classifying clean images, while the conventional adversarialtraining cannot. In accordance with this perspective, we propose a new adversarial training method based on knowledgedistillation using clean-CNN that trained with clean images only as a teacher model. This method transfers the knowledge fromthe clean-CNN and makes feature representations effective for classifying clean images in adversarial training. Our methodoutperforms the conventional adversarial training for both clean images and adversarial examples. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | Convolutional Neural Network / Adversarial Training / Knowledge Distillation / Manifold Hypothesis |
| Paper # | NC2021-44 |
| Date of Issue | 2022-01-14 (NC) |
| Conference Information | |
| Committee | NLP / MICT / MBE / NC |
|---|---|
| Conference Date | 2022/1/21(3days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | Online |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | |
| Chair | Takuji Kosaka(Chukyo Univ.) / Eisuke Hanada(Saga Univ.) / Ryuhei Okuno(Setsunan Univ.) / Rieko Osu(Waseda Univ.) |
| Vice Chair | Akio Tsuneda(Kumamoto Univ.) / Hirokazu Tanaka(Hiroshima City Univ.) / Daisuke Anzai(Nagoya Inst. of Tech.) / Junichi Hori(Niigata Univ.) / Hiroshi Yamakawa(Univ of Tokyo) |
| Secretary | Akio Tsuneda(Kagawa Univ.) / Hirokazu Tanaka(Sojo Univ.) / Daisuke Anzai(Yokohama National Univ.) / Junichi Hori(KISTEC) / Hiroshi Yamakawa(Osaka Electro-Communication Univ) |
| Assistant | Hideyuki Kato(Oita Univ.) / Yuichi Yokoi(Nagasaki Univ.) / Takahiro Ito(Hiroshima City Univ) / Kento Takabayashi(Okayama Pref. Univ.) / Takuya Nishikawa(National Cerebral and Cardiovascular Center Hospital) / Jun Akazawa(Meiji Univ. of Integrative Medicine) / Emi Yuda(Tohoku Univ) / Nobuhiko Wagatsuma(Toho Univ.) / Tomoki Kurikawa(KMU) |
| Paper Information | |
| Registration To | Technical Committee on Nonlinear Problems / Technical Committee on Healthcare and Medical Information Communication Technology / Technical Committee on ME and Bio Cybernetics / Technical Committee on Neurocomputing |
|---|---|
| Language | JPN |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Adversarial Training with Knowledge Distillation considering Intermediate Feature Representation in CNNs |
| Sub Title (in English) | |
| Keyword(1) | Convolutional Neural Network |
| Keyword(2) | Adversarial Training |
| Keyword(3) | Knowledge Distillation |
| Keyword(4) | Manifold Hypothesis |
| 1st Author's Name | Hikaru Higuchi |
| 1st Author's Affiliation | The University of Electro-Communications(The Univ. of Electro-Communications) |
| 2nd Author's Name | Satoshi Suzuki |
| 2nd Author's Affiliation | NTT Computer and Data Science Laboratories, NTT Corporation(former NTT) |
| 3rd Author's Name | Hayaru Shouno |
| 3rd Author's Affiliation | The University of Electro-Communications(The Univ. of Electro-Communications) |
| Date | 2022-01-23 |
| Paper # | NC2021-44 |
| Volume (vol) | vol.121 |
| Number (no) | NC-338 |
| Page | pp.pp.59-64(NC), |
| #Pages | 6 |
| Date of Issue | 2022-01-14 (NC) |