| Presentation | 2021-10-09 Cross-modal CycleGAN for Low-Resource Anime Style Face Translation Shiping Deng, Kaoru Uchida, |
|---|---|
| PDF Download Page |  PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | Human face to anime face translation has attracted the attention of many researchers in recentyears, and various works have achieved high-quality style transfer on conventional tasks. However, existing works often have fatal shortcomings when the target domain training data is heavilyinsufficient, which is named as imbalanced (low-resource) setting. Here the low-resource task, generally means there is no sufficient data on the training dataset compared with the conventionaltask, e.g. the training data size is fewer than 100. To solve this problem, we propose a multimodallow-resource translation model for a specific style. Based on the cyclic adversarial networkand class activation map, we import semantic modality to enhance data information and attentionmodules, which will help our model focus more on the important areas distinguishing the sourcedomain from the target domain. Unlike the previous unsupervised learning of single modality, ourmodel successfully completes image translation in cross-modal situations by importing pre-trainedtext-image alignment model. In addition, the use of an asymmetric structure speeds up training andflexibly generates images of the target domain. The experimental results show that our method hassuperiority in low-resource settings compared with the existing work of the same type. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | Data-imbalanced / style translation / deep neural network |
| Paper # | PRMU2021-18 |
| Date of Issue | 2021-10-01 (PRMU) |
| Conference Information | |
| Committee | PRMU |
|---|---|
| Conference Date | 2021/10/8(2days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | Online |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | Processes and technologies to make research more efficient |
| Chair | Seiichi Uchida(Kyushu Univ.) |
| Vice Chair | Masakazu Iwamura(Osaka Pref. Univ.) / Mitsuru Anpai(Denso IT Lab.) |
| Secretary | Masakazu Iwamura(NTT) / Mitsuru Anpai(Tottori Univ.) |
| Assistant | Kouta Yamaguchi(CyberAgent) / Yusuke Matsui(Univ. of Tokyo) |
| Paper Information | |
| Registration To | Technical Committee on Pattern Recognition and Media Understanding |
|---|---|
| Language | ENG |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Cross-modal CycleGAN for Low-Resource Anime Style Face Translation |
| Sub Title (in English) | |
| Keyword(1) | Data-imbalanced |
| Keyword(2) | style translation |
| Keyword(3) | deep neural network |
| 1st Author's Name | Shiping Deng |
| 1st Author's Affiliation | Hosei University/University of Science and Technology of China(Hosei Univ./USTC) |
| 2nd Author's Name | Kaoru Uchida |
| 2nd Author's Affiliation | Hosei University(Hosei Univ.) |
| Date | 2021-10-09 |
| Paper # | PRMU2021-18 |
| Volume (vol) | vol.121 |
| Number (no) | PRMU-192 |
| Page | pp.pp.11-16(PRMU), |
| #Pages | 6 |
| Date of Issue | 2021-10-01 (PRMU) |