| Presentation | 2022-10-25 Comparison of the Coverage Indicators of Evaluation Data for the Convolutional Neural Networks Yuto Yokoyama, Kozo Okano, Shinpei Ogata, Shin Nakazima, |
|---|---|
| PDF Download Page |  PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | Neuron Coverage (NC) was proposed as a measure to quantify the usefulness of evaluation data against Deep Neural Networks (DNNs): the larger the value of NC, the better the coverage of the evaluation data. Since NC is inappropriate for such a metric in some cases, DeepGauge was proposed to overcome those deficiencies. In this study, we conducted a series of experiments to examine how the two metrics differ when DNN models are tested in terms of label imbalances. We found that (1) the NC is less sensitive and thus is not appropriate as the test metrics, which is in accordance with existing studies, and (2) some DeepGauge indices are appropriately sensitive in certain cases. We also found that those DeepGauge indices do not show any difference for the case of calculating the indices for CNNs, whether the neurons in the convolution and pooling layers are considered or not. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | CNN / NeuronCoverage / DeepGauge |
| Paper # | SS2022-27,DC2022-33 |
| Date of Issue | 2022-10-18 (SS, DC) |
| Conference Information | |
| Committee | DC / SS |
|---|---|
| Conference Date | 2022/10/25(1days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | |
| Chair | Tatsuhiro Tsuchiya(Osaka Univ.) / Kozo Okano(Shinshu Univ.) |
| Vice Chair | Toshinori Hosokawa(Nihon Univ.) / Yoshiki Higo(Osaka Univ.) |
| Secretary | Toshinori Hosokawa(Nihon Univ.) / Yoshiki Higo(Chiba Univ.) |
| Assistant | / Shinsuke Matsumoto(Osaka Univ.) |
| Paper Information | |
| Registration To | Technical Committee on Dependable Computing / Technical Committee on Software Science |
|---|---|
| Language | JPN |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Comparison of the Coverage Indicators of Evaluation Data for the Convolutional Neural Networks |
| Sub Title (in English) | |
| Keyword(1) | CNN |
| Keyword(2) | NeuronCoverage |
| Keyword(3) | DeepGauge |
| 1st Author's Name | Yuto Yokoyama |
| 1st Author's Affiliation | Shinshu University(Shinshu Univ.) |
| 2nd Author's Name | Kozo Okano |
| 2nd Author's Affiliation | Shinshu University(Shinshu Univ.) |
| 3rd Author's Name | Shinpei Ogata |
| 3rd Author's Affiliation | Shinshu University(Shinshu Univ.) |
| 4th Author's Name | Shin Nakazima |
| 4th Author's Affiliation | National Institute of Informatics(NII) |
| Date | 2022-10-25 |
| Paper # | SS2022-27,DC2022-33 |
| Volume (vol) | vol.122 |
| Number (no) | SS-225,DC-226 |
| Page | pp.pp.29-34(SS), pp.29-34(DC), |
| #Pages | 6 |
| Date of Issue | 2022-10-18 (SS, DC) |