| Presentation | 2022-10-07 Detection of human boredom from video Yuki Tachikawa, Atsushi Nakazawa, |
|---|---|
| PDF Download Page |  PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | Prediction of individual internal state is an essential element to realize future affective interactive systems. Nevertheless, the user interfaces that uses agents (avatar) are becoming popular in various application fields, it is not clear whether the users prefer the interaction with the agents. If the system can detect the users' boredom with the agents, the system can change the agent's behavior and prevent user's boredom. In this study, we developed the algorithm to recognize the user's `boredom' from facial images. 31 participants were asked to perform a conversational task with the agent on topics that they were supposed to be interested in (e.g. food and events) and not interested in (e.g. geometry and architecture) and their facial videos are taken. For the video, we detected the facial parts and normalized the facial regions from the input images, and obtained optical flow. For the recognition, two types of networks, 2D-CNN and 3D-CNN, were developed. As the result, the recognition rate of the boredom were 60% (2D-CNN) and 54% (3D-CNN) (chance rate = 50%), respectively. Moreover, we trained the same network to identify four types of personalities of the users. As the result, the accuracy were 33% (2D-CNN) and 39% (3D-CNN) (chance rate = 25%), respectively. These results indicated that learning facial expression movements with a CNN can be used to estimate boredom and personality, and visualization of the learned network can be used to estimate which regions are the basis for discrimination. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | facial expression analysis / boredom / personality |
| Paper # | MVE2022-27 |
| Date of Issue | 2022-09-29 (MVE) |
| Conference Information | |
| Committee | MVE / VRSJ-SIG-MR / IPSJ-EC / HI-SIG-DeMO / VRSJ-SIG-CS |
|---|---|
| Conference Date | 2022/10/6(2days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | |
| Chair | Kiyoshi Kiyokawa(NAIST) |
| Vice Chair | Sumaru Niida(KDDI Research) |
| Secretary | Sumaru Niida(NAIST) / (DNP) / (Univ. of ToKyo) / (NTT) |
| Assistant | Hidehiko Shishido(Univ. of Tsukuba) / Atsushi Nakazawa(Kyoto Univ.) / Naoya Tojo(KDDI Research) / Naoki Hagiyama(NTT) |
| Paper Information | |
| Registration To | Technical Committee on Media Experience and Virtual Environment / SIG-MR / Special Interest Group on Entertainment Computing / Special Interest Group on De-vice Media Oriented UI / SIG-CS |
|---|---|
| Language | JPN |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Detection of human boredom from video |
| Sub Title (in English) | |
| Keyword(1) | facial expression analysis |
| Keyword(2) | boredom |
| Keyword(3) | personality |
| 1st Author's Name | Yuki Tachikawa |
| 1st Author's Affiliation | Kyoto University(Kyoto Univ.) |
| 2nd Author's Name | Atsushi Nakazawa |
| 2nd Author's Affiliation | Kyoto University(Kyoto Univ.) |
| Date | 2022-10-07 |
| Paper # | MVE2022-27 |
| Volume (vol) | vol.122 |
| Number (no) | MVE-200 |
| Page | pp.pp.52-56(MVE), |
| #Pages | 5 |
| Date of Issue | 2022-09-29 (MVE) |