| Presentation | 2020-03-06 Generating Video of Cloned Motion from a Short Sequence of Still Images Considering Self-Occlusion of Human Body Tsutsumi Teppei, Kazuaki Nakamura, Seiko Myojin, Naoko Nitta, Noboru Babaguchi, |
|---|---|
| PDF Download Page |  PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | In this paper, we propose a method for generating photo-realistic video in which a person virtually performs a motion that is not performed in the real world. We refer to such video as video of cloned motion, abbreviated as VCM. The proposed method requires only two kinds of source data: a reference video, in which a person A performs some motion, and several target images, which include the whole body of another person B. Using these data, our method generates a VCM in which the person A's motion is re-enacted by the person B's body. There is an existing method that only deforming a single target image to generate a VCM. However, this method has two problems: First, it cannot handle the self-occlusion of the human body. Second, it cannot generate the texture of the body surfaces that are invisible in the target image. Our method copes with the ?rst problem by dividing the human body into multiple parts and deforming the target image part-by-part using part-wise 2D a?ne transforms. Moreover, to cope with the second problem, we use a convolutional neural network to combine multiple VCMs each of which is separately generated from a single target image. Our experimental results show the robustness of the proposed method to the above two problems. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | reference video / target image / video of cloned motion / self-occlusion / 2D affine transform / convolutional neural network |
| Paper # | IMQ2019-58,IE2019-140,MVE2019-79 |
| Date of Issue | 2020-02-27 (IMQ, IE, MVE) |
| Conference Information | |
| Committee | IE / IMQ / MVE / CQ |
|---|---|
| Conference Date | 2020/3/5(2days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | Kyushu Institute of Technology |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | |
| Chair | Hideaki Kimata(NTT) / Toshiya Nakaguchi(Chiba Univ.) / Kenji Mase(Nagoya Univ.) / Hideyuki Shimonishi(NEC) |
| Vice Chair | Kazuya Kodama(NII) / Keita Takahashi(Nagoya Univ.) / Mitsuru Maeda(Canon) / Kenya Uomori(Osaka Univ.) / Masayuki Ihara(NTT) / Jun Okamoto(NTT) / Takefumi Hiraguri(Nippon Inst. of Tech.) |
| Secretary | Kazuya Kodama(NTT) / Keita Takahashi(NHK) / Mitsuru Maeda(Shizuoka Univ.) / Kenya Uomori(Sony Semiconductor Solutions) / Masayuki Ihara(Nagoya Univ.) / Jun Okamoto(NTT) / Takefumi Hiraguri(Nippon Inst. of Tech.) |
| Assistant | Kyohei Unno(KDDI Research) / Norishige Fukushima(Nagoya Inst. of Tech.) / Hiroaki Kudo(Nagoya Univ.) / Masaru Tsuchida(NTT) / Keita Hirai(Chiba Univ.) / Satoshi Nishiguchi(Oosaka Inst. of Tech.) / Masanori Yokoyama(NTT) / Shogo Fukushima(Univ. of ToKyo) / Chikara Sasaki(KDDI Research) / Yoshiaki Nishikawa(NEC) / Takuto Kimura(NTT) |
| Paper Information | |
| Registration To | Technical Committee on Image Engineering / Technical Committee on Image Media Quality / Technical Committee on Media Experience and Virtual Environment / Technical Committee on Communication Quality |
|---|---|
| Language | JPN |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Generating Video of Cloned Motion from a Short Sequence of Still Images Considering Self-Occlusion of Human Body |
| Sub Title (in English) | |
| Keyword(1) | reference video |
| Keyword(2) | target image |
| Keyword(3) | video of cloned motion |
| Keyword(4) | self-occlusion |
| Keyword(5) | 2D affine transform |
| Keyword(6) | convolutional neural network |
| 1st Author's Name | Tsutsumi Teppei |
| 1st Author's Affiliation | Osaka University(Osaka Univ.) |
| 2nd Author's Name | Kazuaki Nakamura |
| 2nd Author's Affiliation | Osaka University(Osaka Univ.) |
| 3rd Author's Name | Seiko Myojin |
| 3rd Author's Affiliation | Osaka University(Osaka Univ.) |
| 4th Author's Name | Naoko Nitta |
| 4th Author's Affiliation | Osaka University(Osaka Univ.) |
| 5th Author's Name | Noboru Babaguchi |
| 5th Author's Affiliation | Osaka University(Osaka Univ.) |
| Date | 2020-03-06 |
| Paper # | IMQ2019-58,IE2019-140,MVE2019-79 |
| Volume (vol) | vol.119 |
| Number (no) | IMQ-454,IE-456,MVE-457 |
| Page | pp.pp.221-226(IMQ), pp.221-226(IE), pp.221-226(MVE), |
| #Pages | 6 |
| Date of Issue | 2020-02-27 (IMQ, IE, MVE) |