| Presentation | 2017-09-25 A Memory Reduction with Neuron Pruning for a Binarized Deep Convolutional Neural Network: Its FPGA Realization Tomoya Fujii, Shimpei Sato, Hiroki Nakahara, |
|---|---|
| PDF Download Page |  PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | For a pre-trained deep convolutional neural network (CNN) for an embedded system, a high-speed and a low power consumption are required. In the former of the CNN, it consists of convolutional layers, whilein the latter, it consists of fully connection layers. In the convolutionallayer, the multiply accumulation operation is a bottleneck, while the fullyconnection layer, the memory access is a bottleneck. The binarized CNNhas been proposed to realize many multiply accumulation circuit on theFPGA, thus, the convolutional layer can be done with high-seed operation. However, even if we apply the binarization to the fully connection layer, theamount of memory was still bottleneck. In this paper, we propose a neuron pruning technique which eliminates almost part of the weight memory, and apply it to the fully connection layer on the binarized CNN. In that case, since the weight memory is realized by an on-chip memory on the FPGA, it achieves a high speed memory access. To further reduce the memory size, we apply the retraining the CNN after neuron pruning. In this paper, we propose a sequential-input parallel-output fully connection layer circuit for the binarized fully connection layer, while propose a streaming circuit for the binarized 2D convolutional layer. The experimental results showed that, by the neuron pruning, as for the fully connected layer on the VGG-11 CNN, the number of neurons was reduced by 60.2%, and the amount of memory was reduced by 83% with keeping the 99% baseline recognition accuracy. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | CNN / FPGA / Memory Reduction / Pruning |
| Paper # | RECONF2017-26 |
| Date of Issue | 2017-09-18 (RECONF) |
| Conference Information | |
| Committee | RECONF |
|---|---|
| Conference Date | 2017/9/25(2days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | DWANGO Co., Ltd. |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | Reconfigurable Systems, etc. |
| Chair | Masato Motomura(Hokkaido Univ.) |
| Vice Chair | Yuichiro Shibata(Nagasaki Univ.) / Kentaro Sano(Tohoku Univ.) |
| Secretary | Yuichiro Shibata(Hiroshima City Univ.) / Kentaro Sano(e-trees.Japan) |
| Assistant | Yuuki Kobayashi(NEC) / Hiroki Nakahara(Tokyo Inst. of Tech.) |
| Paper Information | |
| Registration To | Technical Committee on Reconfigurable Systems |
|---|---|
| Language | JPN |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | A Memory Reduction with Neuron Pruning for a Binarized Deep Convolutional Neural Network: Its FPGA Realization |
| Sub Title (in English) | |
| Keyword(1) | CNN |
| Keyword(2) | FPGA |
| Keyword(3) | Memory Reduction |
| Keyword(4) | Pruning |
| 1st Author's Name | Tomoya Fujii |
| 1st Author's Affiliation | Tokyo Institute of Technology(Tokyo Inst. of Tech.) |
| 2nd Author's Name | Shimpei Sato |
| 2nd Author's Affiliation | Tokyo Institute of Technology(Tokyo Inst. of Tech.) |
| 3rd Author's Name | Hiroki Nakahara |
| 3rd Author's Affiliation | Tokyo Institute of Technology(Tokyo Inst. of Tech.) |
| Date | 2017-09-25 |
| Paper # | RECONF2017-26 |
| Volume (vol) | vol.117 |
| Number (no) | RECONF-221 |
| Page | pp.pp.25-30(RECONF), |
| #Pages | 6 |
| Date of Issue | 2017-09-18 (RECONF) |