| Presentation | 2018-08-01 Optimization of Numerical Expression in CNN using Genetic Algorithm Wakana Nogami, Tsutomu Ikegami, Shin-ichi O'uchi, Ryosei Takano, Yuma Kishi, Tomohiro Kudoh, |
|---|---|
| PDF Download Page |  PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | The accuracy of image recognition by the convolutional neural network (CNN) has been improving year by year, and the models are becoming more complicated and larger. One way to reduce the model size is to use a low bit-width numerical expression. There are many types of research to reduce the bit-width by using floating-point, fixed-point, ternary, and binary arithmetics, and so on. They are aiming at (1) simplifying the computation (2) by introducing a less-bit arithmetic (3) to keep the image recognition accuracy. Considering ease of computation, it is appropriate to use floating-point and fixed-point. Therefore, we introduced a variable bin size quantization and found the appropriate numerical expression from the viewpoint of low bit-width and high accuracy by optimizing the bin size using a genetic algorithm. In this study, we targeted on quantization of trained parameters at inference. As a result, we succeeded in finding a numerical expression that can give higher Top-1 Accuracy than when using fixed-point or floating-point type for our CNN models. The numerical expression is relatively similar to fixed-point type. We also found that by using this numerical expression, it is possible to reduce the bit-width down to 3-bit without decreasing the accuracy. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | CNN / Adaptive Quantization / Genetic Algorithm / Numerical Expression / Optimization |
| Paper # | CPSY2018-26 |
| Date of Issue | 2018-07-23 (CPSY) |
| Conference Information | |
| Committee | CPSY / DC / IPSJ-ARC |
|---|---|
| Conference Date | 2018/7/30(3days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | Kumamoto City International Center |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | Parallel, Distributed and Cooperative Processing Systems and Dependable Computing |
| Chair | Koji Nakano(Hiroshima Univ.) / Satoshi Fukumoto(Tokyo Metropolitan Univ.) / Masahiro Goshima(NII) |
| Vice Chair | Hidetsugu Irie(Univ. of Tokyo) / Takashi Miyoshi(Fujitsu) / Hiroshi Takahashi(Ehime Univ.) |
| Secretary | Hidetsugu Irie(Utsunomiya Univ.) / Takashi Miyoshi(Hokkaido Univ.) / Hiroshi Takahashi(Tokyo Inst. of Tech.) / (Nihon Univ.) |
| Assistant | Yasuaki Ito(Hiroshima Univ.) / Tomoaki Tsumura(Nagoya Inst. of Tech.) |
| Paper Information | |
| Registration To | Technical Committee on Computer Systems / Technical Committee on Dependable Computing / Special Interest Group on System Architecture |
|---|---|
| Language | JPN |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Optimization of Numerical Expression in CNN using Genetic Algorithm |
| Sub Title (in English) | |
| Keyword(1) | CNN |
| Keyword(2) | Adaptive Quantization |
| Keyword(3) | Genetic Algorithm |
| Keyword(4) | Numerical Expression |
| Keyword(5) | Optimization |
| 1st Author's Name | Wakana Nogami |
| 1st Author's Affiliation | The University of Tokyo/National Institute of Advanced Industrial And Technology(Tokyo Univ./AIST) |
| 2nd Author's Name | Tsutomu Ikegami |
| 2nd Author's Affiliation | National Institute of Advanced Industrial And Technology(AIST) |
| 3rd Author's Name | Shin-ichi O'uchi |
| 3rd Author's Affiliation | National Institute of Advanced Industrial And Technology(AIST) |
| 4th Author's Name | Ryosei Takano |
| 4th Author's Affiliation | National Institute of Advanced Industrial And Technology(AIST) |
| 5th Author's Name | Yuma Kishi |
| 5th Author's Affiliation | The University of Tokyo/National Institute of Advanced Industrial And Technology(Tokyo Univ./AIST) |
| 6th Author's Name | Tomohiro Kudoh |
| 6th Author's Affiliation | The University of Tokyo/National Institute of Advanced Industrial And Technology(Tokyo Univ./AIST) |
| Date | 2018-08-01 |
| Paper # | CPSY2018-26 |
| Volume (vol) | vol.118 |
| Number (no) | CPSY-165 |
| Page | pp.pp.193-198(CPSY), |
| #Pages | 6 |
| Date of Issue | 2018-07-23 (CPSY) |