IEICE Technical Committee Submission System
Conference Paper's Information
Online Proceedings
[Sign in]
Tech. Rep. Archives
 Go Top Page Go Previous   [Japanese] / [English] 

Paper Abstract and Keywords
Presentation 2020-03-06 11:35
Generating Video of Cloned Motion from a Short Sequence of Still Images Considering Self-Occlusion of Human Body
Tsutsumi Teppei, Kazuaki Nakamura, Seiko Myojin, Naoko Nitta, Noboru Babaguchi (Osaka Univ.) IMQ2019-58 IE2019-140 MVE2019-79
Abstract (in Japanese) (See Japanese page) 
(in English) In this paper, we propose a method for generating photo-realistic video in which a person virtually performs a motion that is not performed in the real world. We refer to such video as video of cloned motion, abbreviated as VCM. The proposed method requires only two kinds of source data: a reference video, in which a person A performs some motion, and several target images, which include the whole body of another person B. Using these data, our method generates a VCM in which the person A's motion is re-enacted by the person B's body. There is an existing method that only deforming a single target image to generate a VCM. However, this method has two problems: First, it cannot handle the self-occlusion of the human body. Second, it cannot generate the texture of the body surfaces that are invisible in the target image. Our method copes with the first problem by dividing the human body into multiple parts and deforming the target image part-by-part using part-wise 2D affine transforms. Moreover, to cope with the second problem, we use a convolutional neural network to combine multiple VCMs each of which is separately generated from a single target image. Our experimental results show the robustness of the proposed method to the above two problems.
Keyword (in Japanese) (See Japanese page) 
(in English) reference video / target image / video of cloned motion / self-occlusion / 2D affine transform / convolutional neural network / /  
Reference Info. IEICE Tech. Rep., vol. 119, no. 457, MVE2019-79, pp. 221-226, March 2020.
Paper # MVE2019-79 
Date of Issue 2020-02-27 (IMQ, IE, MVE) 
ISSN Print edition: ISSN 0913-5685  Online edition: ISSN 2432-6380
All rights are reserved and no part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy, recording, or any information storage and retrieval system, without permission in writing from the publisher. Notwithstanding, instructors are permitted to photocopy isolated articles for noncommercial classroom use without fee. (License No.: 10GA0019/12GB0052/13GB0056/17GB0034/18GB0034)
Download PDF IMQ2019-58 IE2019-140 MVE2019-79

Conference Information
Committee IE IMQ MVE CQ  
Conference Date 2020-03-05 - 2020-03-06 
Place (in Japanese) (See Japanese page) 
Place (in English) Kyushu Institute of Technology 
Topics (in Japanese) (See Japanese page) 
Topics (in English)  
Paper Information
Registration To MVE 
Conference Code 2020-03-IE-IMQ-MVE-CQ 
Language Japanese 
Title (in Japanese) (See Japanese page) 
Sub Title (in Japanese) (See Japanese page) 
Title (in English) Generating Video of Cloned Motion from a Short Sequence of Still Images Considering Self-Occlusion of Human Body 
Sub Title (in English)  
Keyword(1) reference video  
Keyword(2) target image  
Keyword(3) video of cloned motion  
Keyword(4) self-occlusion  
Keyword(5) 2D affine transform  
Keyword(6) convolutional neural network  
1st Author's Name Tsutsumi Teppei  
1st Author's Affiliation Osaka University (Osaka Univ.)
2nd Author's Name Kazuaki Nakamura  
2nd Author's Affiliation Osaka University (Osaka Univ.)
3rd Author's Name Seiko Myojin  
3rd Author's Affiliation Osaka University (Osaka Univ.)
4th Author's Name Naoko Nitta  
4th Author's Affiliation Osaka University (Osaka Univ.)
5th Author's Name Noboru Babaguchi  
5th Author's Affiliation Osaka University (Osaka Univ.)
6th Author's Name  
6th Author's Affiliation ()
7th Author's Name  
7th Author's Affiliation ()
8th Author's Name  
8th Author's Affiliation ()
9th Author's Name  
9th Author's Affiliation ()
10th Author's Name  
10th Author's Affiliation ()
11th Author's Name  
11th Author's Affiliation ()
12th Author's Name  
12th Author's Affiliation ()
13th Author's Name  
13th Author's Affiliation ()
14th Author's Name  
14th Author's Affiliation ()
15th Author's Name  
15th Author's Affiliation ()
16th Author's Name  
16th Author's Affiliation ()
17th Author's Name  
17th Author's Affiliation ()
18th Author's Name  
18th Author's Affiliation ()
19th Author's Name  
19th Author's Affiliation ()
20th Author's Name  
20th Author's Affiliation ()
Date Time 2020-03-06 11:35:00 
Presentation Time 25 
Registration for MVE 
Paper # IEICE-IMQ2019-58,IEICE-IE2019-140,IEICE-MVE2019-79 
Volume (vol) IEICE-119 
Number (no) no.454(IMQ), no.456(IE), no.457(MVE) 
Page pp.221-226 
#Pages IEICE-6 
Date of Issue IEICE-IMQ-2020-02-27,IEICE-IE-2020-02-27,IEICE-MVE-2020-02-27 

[Return to Top Page]

[Return to IEICE Web Page]

The Institute of Electronics, Information and Communication Engineers (IEICE), Japan