Presentation 2017-01-24
A Memory Reduction with Neuron Pruning for a Convolutional Neural Network: Its FPGA Realization
Tomoya Fujii, Simpei Sato, Hiroki Nakahara, Masato Motomura,
PDF Download Page PDF download Page Link
Abstract(in Japanese) (See Japanese page)
Abstract(in English) For a pre-trained deep convolutional neural network (CNN) aim at an embedded system, a high-speed and a low power consumption are required. In the former of the CNN, it consists of convolutional layers, while in the latter, it consists of fully connection layers. In the convolutional layer, the multipliy accumulation operation is a bottleneck, while the fully connection layer, the memory access is a bottleneck. In this paper, we propose a neuron pruning technique which eliminates almost part of the weight memory. In that case, it is realized by an on-chip memory on the FPGA. Thus, it acheives a high speed memory access. In this paper, we propose a sequential-input parallel-output fully connection layer circuit. The experimental results showed that, by the neuron pruning, as for the fully connected layer on the VGG-11 CNN, the number of neurons was reduced by 76.4% with keeping the 99% accuracy. We implemented the fully connected layers on the Digilent Inc. NetFPGA-1G-CML FPGA board. Comparison with the CPU (ARM Cortex A15 processor) and the GPU (Jetson TK1 Kepler), as for a delay time, the FPGA was 219.0 times faster than the CPU and 12.5 times faster than the GPU. Also, a performance per power efficiency was 87.69 times better than CPU and 12.51 times better than GPU.
Keyword(in Japanese) (See Japanese page)
Keyword(in English) Convolutinal Neural Network / FPGA / Pruning
Paper # VLD2016-79,CPSY2016-115,RECONF2016-60
Date of Issue 2017-01-16 (VLD, CPSY, RECONF)

Conference Information
Committee CPSY / RECONF / VLD / IPSJ-SLDM / IPSJ-ARC
Conference Date 2017/1/23(3days)
Place (in Japanese) (See Japanese page)
Place (in English) Hiyoshi Campus, Keio Univ.
Topics (in Japanese) (See Japanese page)
Topics (in English) FPGA Applications, etc
Chair Yasuhiko Nakashima(NAIST) / Minoru Watanabe(Shizuoka Univ.) / Takashi Takenana(NEC) / Masahiro Fukui(Ritsumeikan Univ.) / 五島 正裕(NII)
Vice Chair Koji Nakano(Hiroshima Univ.) / Hidetsugu Irie(Univ. of Tokyo) / Masato Motomura(Hokkaido Univ.) / Yuichiro Shibata(Nagasaki Univ.) / Hiroyuki Ochi(Ritsumeikan Univ.)
Secretary Koji Nakano(Fujitsu Labs.) / Hidetsugu Irie(NII) / Masato Motomura(Univ. of Tsukuba) / Yuichiro Shibata(Hiroshima City Univ.) / Hiroyuki Ochi(Fujitsu Labs.) / (Hiroshima City Univ.) / (Sharp)
Assistant Takeshi Ohkawa(Utsunomiya Univ.) / Shinya Takameda(Hokkaido Univ.) / Takefumi Miyoshi(e-trees.Japan) / Yuuki Kobayashi(NEC) / Parizy Matthieu(Fujitsu Labs.)

Paper Information
Registration To Technical Committee on Computer Systems / Technical Committee on Reconfigurable Systems / Technical Committee on VLSI Design Technologies / Special Interest Group on System and LSI Design Methodology / Special Interest Group on System Architecture
Language JPN
Title (in Japanese) (See Japanese page)
Sub Title (in Japanese) (See Japanese page)
Title (in English) A Memory Reduction with Neuron Pruning for a Convolutional Neural Network: Its FPGA Realization
Sub Title (in English)
Keyword(1) Convolutinal Neural Network
Keyword(2) FPGA
Keyword(3) Pruning
1st Author's Name Tomoya Fujii
1st Author's Affiliation Tokyo Institute of Technology(Tokyo Tech)
2nd Author's Name Simpei Sato
2nd Author's Affiliation Tokyo Institute of Technology(Tokyo Tech)
3rd Author's Name Hiroki Nakahara
3rd Author's Affiliation Tokyo Institute of Technology(Tokyo Tech)
4th Author's Name Masato Motomura
4th Author's Affiliation Hokkaido University(Hokkaido univ.)
Date 2017-01-24
Paper # VLD2016-79,CPSY2016-115,RECONF2016-60
Volume (vol) vol.116
Number (no) VLD-415,CPSY-416,RECONF-417
Page pp.pp.55-60(VLD), pp.55-60(CPSY), pp.55-60(RECONF),
#Pages 6
Date of Issue 2017-01-16 (VLD, CPSY, RECONF)