Paper Abstract and Keywords |
Presentation |
2018-08-01 15:15
Optimization of Numerical Expression in CNN using Genetic Algorithm Wakana Nogami (Tokyo Univ./AIST), Tsutomu Ikegami, Shin-ichi O'uchi, Ryosei Takano (AIST), Yuma Kishi, Tomohiro Kudoh (Tokyo Univ./AIST) CPSY2018-26 |
Abstract |
(in Japanese) |
(See Japanese page) |
(in English) |
The accuracy of image recognition by the convolutional neural network (CNN) has been improving year by year, and the models are becoming more complicated and larger. One way to reduce the model size is to use a low bit-width numerical expression. There are many types of research to reduce the bit-width by using floating-point, fixed-point, ternary, and binary arithmetics, and so on. They are aiming at (1) simplifying the computation (2) by introducing a less-bit arithmetic (3) to keep the image recognition accuracy. Considering ease of computation, it is appropriate to use floating-point and fixed-point. Therefore, we introduced a variable bin size quantization and found the appropriate numerical expression from the viewpoint of low bit-width and high accuracy by optimizing the bin size using a genetic algorithm. In this study, we targeted on quantization of trained parameters at inference. As a result, we succeeded in finding a numerical expression that can give higher Top-1 Accuracy than when using fixed-point or floating-point type for our CNN models. The numerical expression is relatively similar to fixed-point type. We also found that by using this numerical expression, it is possible to reduce the bit-width down to 3-bit without decreasing the accuracy. |
Keyword |
(in Japanese) |
(See Japanese page) |
(in English) |
CNN / Adaptive Quantization / Genetic Algorithm / Numerical Expression / Optimization / / / |
Reference Info. |
IEICE Tech. Rep., vol. 118, no. 165, CPSY2018-26, pp. 193-198, July 2018. |
Paper # |
CPSY2018-26 |
Date of Issue |
2018-07-23 (CPSY) |
ISSN |
Online edition: ISSN 2432-6380 |
Copyright and reproduction |
All rights are reserved and no part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy, recording, or any information storage and retrieval system, without permission in writing from the publisher. Notwithstanding, instructors are permitted to photocopy isolated articles for noncommercial classroom use without fee. (License No.: 10GA0019/12GB0052/13GB0056/17GB0034/18GB0034) |
Download PDF |
CPSY2018-26 |
|