| Presentation | 2023-05-18 Discriminating between fake and real gestures in automatic gesture generation Geng Mu, Nosh Kaneko, Kazuhiko Sumi, |
|---|---|
| PDF Download Page |  PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | In recent years, gestures play a crucial role in communication with anthropomorphized agents and robots. The use of gestures by these artificial entities during conversations with humans is expected to facilitate more natural communication. However, as technologies for automatically generating these gestures advance, there is an increasing risk of generated gestures being exploited for malicious purposes. In this context, this study proposes a method for distinguishing between automatically generated (fake) gestures and human-performed (real) gestures.Specifically, we have constructed a dataset consisting of fake gestures generated using existing data-driven gesture generation methods and real gestures obtained through pose estimation. By training a skeleton-based action recognition model on this dataset, we propose a method for distinguishing between fake and real gestures. Utilizing a learning dataset generated by three different fake gesture generation methods from 25 speakers with different speaking styles and 251 hours of audio, text, and image information, we trained the action recognition network as a real/fake discrimination problem. This approach successfully detected unlearned fake gestures generated by other algorithms with an accuracy of 99.72%. This approach is anticipated to mitigate the risk of misuse arising from the advancements in gesture generation technology, providing a foundation for safer and more authentic interactions in human-robot communication. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | Deepfake detection / Gesture generation / Generative models / Human action recognition |
| Paper # | PRMU2023-5 |
| Date of Issue | 2023-05-11 (PRMU) |
| Conference Information | |
| Committee | PRMU / IPSJ-CVIM |
|---|---|
| Conference Date | 2023/5/18(2days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | |
| Chair | Seiichi Uchida(Kyushu Univ.) |
| Vice Chair | Takuya Funatomi(NAIST) / Mitsuru Anpai(Denso IT Lab.) |
| Secretary | Takuya Funatomi(CyberAgent) / Mitsuru Anpai(Univ. of Tokyo) |
| Assistant | Nakamasa Inoue(Tokyo Inst. of Tech.) / Yasutomo Kawanishi(Riken) |
| Paper Information | |
| Registration To | Technical Committee on Pattern Recognition and Media Understanding / Special Interest Group on Computer Vision and Image Media |
|---|---|
| Language | JPN |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Discriminating between fake and real gestures in automatic gesture generation |
| Sub Title (in English) | |
| Keyword(1) | Deepfake detection |
| Keyword(2) | Gesture generation |
| Keyword(3) | Generative models |
| Keyword(4) | Human action recognition |
| 1st Author's Name | Geng Mu |
| 1st Author's Affiliation | Aoyama Gakuin University(AGU) |
| 2nd Author's Name | Nosh Kaneko |
| 2nd Author's Affiliation | Aoyama Gakuin University(AGU) |
| 3rd Author's Name | Kazuhiko Sumi |
| 3rd Author's Affiliation | Aoyama Gakuin University(AGU) |
| Date | 2023-05-18 |
| Paper # | PRMU2023-5 |
| Volume (vol) | vol.123 |
| Number (no) | PRMU-30 |
| Page | pp.pp.22-26(PRMU), |
| #Pages | 5 |
| Date of Issue | 2023-05-11 (PRMU) |