| Presentation | 2018-06-28 Sound recovery using vibration mode of an object in video Yohei Fuse, Yusuke Yasumi, Tetsuya Takiguchi, |
|---|---|
| PDF Download Page |  PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | When a sound hits an object, it causes the surface of the object to vibrate. Some research has been carried out on the recovering of sounds by extracting the vibrations seen on video images. This research is expected to be applied in the field of surveillance and security because sounds can be recorded from relatively far away. The vibration of objects due to sound is so fast and minute that it is invisible. However, it is possible to observe such changes in objects by using the high-speed video as the movement of each pixel by using a complex steerable pyramid. In the conventional method, the movements of all pixels are added together to recover the sound. So it is possible that some noise source vibrations are mixed because there are some pixels that move independently of the sound source being focused upon. In this paper, we propose a sound recovery method focusing on the vibration modes of the object associated with the frequency. The vibrating parts of objects are different depending on the material, shape and frequency. The vibration is composed of some normal vibrations, and each has different loops and nodes. We confirm which part of the object is vibrating for each frequency of the sound, and recover the sound using a filter based on the response of the object. Which part is vibrating is confirmed from the amplitude response of each pixel when the signal of that frequency is the largest. This response and the reliability of the signal of each pixel are multiplied to each pixel as a filter. We recovered sounds from several objects in videos and ascertained the effectiveness of the method. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | sound recovery / object vibration / high-speed video / vibration mode |
| Paper # | PRMU2018-25,SP2018-5 |
| Date of Issue | 2018-06-21 (PRMU, SP) |
| Conference Information | |
| Committee | PRMU / SP |
|---|---|
| Conference Date | 2018/6/28(2days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | |
| Chair | Shinichi Sato(NII) / Yoichi Yamashita(Ritsumeikan Univ.) |
| Vice Chair | Yoshihisa Ijiri(Omron) / Toru Tamaki(Hiroshima Univ.) / Akinobu Ri(Nagoya Inst. of Tech.) |
| Secretary | Yoshihisa Ijiri(NEC) / Toru Tamaki(Osaka Univ.) / Akinobu Ri(Kyoto Univ.) |
| Assistant | Go Irie(NTT) / Yoshitaka Ushiku(Univ. of Tokyo) / Tomoki Koriyama(Tokyo Inst. of Tech.) / Satoshi Kobashikawa(NTT) |
| Paper Information | |
| Registration To | Technical Committee on Pattern Recognition and Media Understanding / Technical Committee on Speech |
|---|---|
| Language | JPN |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Sound recovery using vibration mode of an object in video |
| Sub Title (in English) | |
| Keyword(1) | sound recovery |
| Keyword(2) | object vibration |
| Keyword(3) | high-speed video |
| Keyword(4) | vibration mode |
| 1st Author's Name | Yohei Fuse |
| 1st Author's Affiliation | Kobe University(Kobe Univ.) |
| 2nd Author's Name | Yusuke Yasumi |
| 2nd Author's Affiliation | Kobe University(Kobe Univ.) |
| 3rd Author's Name | Tetsuya Takiguchi |
| 3rd Author's Affiliation | Kobe University(Kobe Univ.) |
| Date | 2018-06-28 |
| Paper # | PRMU2018-25,SP2018-5 |
| Volume (vol) | vol.118 |
| Number (no) | PRMU-111,SP-112 |
| Page | pp.pp.19-24(PRMU), pp.19-24(SP), |
| #Pages | 6 |
| Date of Issue | 2018-06-21 (PRMU, SP) |